Overview
About $0.5 trillion dollars is spent on research and development each year in the U.S. (Boroush, 2015). Over 10% of this spending comes from colleges and universities. And yet, approximately 33% of articles in the social sciences and 80% in the humanities receive no citations (Larivière, et al., 2009). Despite debates on the actual figures, most scholars agree that publication and readership issues are a major problem in the research world (Evans, 2008).
Part of the problem is academia's reluctance to research topics of interest to people other than their field's editors, reviewers, and self (i.e., "mesearch"), To guide us toward a more relevant research agenda, we undertook a study with 1,000 everyday people to find out what topics they want to see researched. To add some spice to the study, we paired this data with a series of demographic and personality variables to test whether topic interest differed between certain groups of people.
The Study
We surveyed 1,000 people on Amazon MTurk to find out what topics they'd like more research on. Each person surveyed was asked to list 10 topics, for a total of 10,000 research topics. This list was then plugged into a word cloud generator—a visual depiction of how many times each word is mentioned—to provide a visual representation of what types of research people want to see, and also to help guide our research focus.
We also had respondents take a psychology survey to measure their 5 core personality traits, namely extroversion, agreeableness, openness to experience, conscientiousness (i.e., diligent, planned), and neuroticism (i.e., anxious, bad moods), as well as some demographic variables, including gender, age, education, and income. These variables helped us test whether the top research topics of interest differ across people with different personalities and demographics; in essence, do nature or nurture affect what we really want to see researched? Separate word clouds were created for the bottom 25% and top 25% of individuals along each variable (e.g., comparing the most popular topics of the 250 most extroverted people with the 250 most introverted, the 250 youngest to the 250 oldest, etc.).
About $0.5 trillion dollars is spent on research and development each year in the U.S. (Boroush, 2015). Over 10% of this spending comes from colleges and universities. And yet, approximately 33% of articles in the social sciences and 80% in the humanities receive no citations (Larivière, et al., 2009). Despite debates on the actual figures, most scholars agree that publication and readership issues are a major problem in the research world (Evans, 2008).
Part of the problem is academia's reluctance to research topics of interest to people other than their field's editors, reviewers, and self (i.e., "mesearch"), To guide us toward a more relevant research agenda, we undertook a study with 1,000 everyday people to find out what topics they want to see researched. To add some spice to the study, we paired this data with a series of demographic and personality variables to test whether topic interest differed between certain groups of people.
The Study
We surveyed 1,000 people on Amazon MTurk to find out what topics they'd like more research on. Each person surveyed was asked to list 10 topics, for a total of 10,000 research topics. This list was then plugged into a word cloud generator—a visual depiction of how many times each word is mentioned—to provide a visual representation of what types of research people want to see, and also to help guide our research focus.
We also had respondents take a psychology survey to measure their 5 core personality traits, namely extroversion, agreeableness, openness to experience, conscientiousness (i.e., diligent, planned), and neuroticism (i.e., anxious, bad moods), as well as some demographic variables, including gender, age, education, and income. These variables helped us test whether the top research topics of interest differ across people with different personalities and demographics; in essence, do nature or nurture affect what we really want to see researched? Separate word clouds were created for the bottom 25% and top 25% of individuals along each variable (e.g., comparing the most popular topics of the 250 most extroverted people with the 250 most introverted, the 250 youngest to the 250 oldest, etc.).
Results
A variety of topics were listed. The word cloud generated from these topics is presented below, followed by a list of the Top 20.
A variety of topics were listed. The word cloud generated from these topics is presented below, followed by a list of the Top 20.
Top 20 Research Topics
- People
- Effects (i.e., testing causality, not just polls or correlations)
- Climate change
- Cancer
- Health
- Food
- Children
- Social media
- Life
- Music
- Energy
- Work
- Politics
- Depression
- Education
- Technology
- Space
- Marijuana
- Income
- Video games
Interesting! But did the results differ between different groups of people? Well, to borrow an old cliché, “We may not be so different, you and I.” Less than a quarter of the top 20 words differed significantly in frequency across people of different personality traits and demographics, and those that did differ were small effect sizes at best. The results were even less significant for the Top 10 topics, our pre-registered target for analysis. Nevertheless, there were a few small yet interesting differences, which we’ve highlighted in the statistical tables for each personality trait and demographic category.
First, there were differences with respect to which topics made the “Top 20” for each sub-sample. For example, the topic “anxiety” made introverts’ Top 20 list, but not extroverts’ (we listed any topic that made either top 20 list). Introverts also tend to want a bit more research on music and education than extroverts.
“Agreeable” people would like to see some more research on cancer, climate change, and children, along with depression and mental health (saving the world and the stress from it?) than those with a “tell it like it is” attitude.
Women want a bit more research on children and school, whereas men prefer more research on video games and sports.
Graduates with the highest levels of education want more research on “effects” showing causality, as opposed to mere correlations or polls, whereas those with less education prefer more research on cancer.
And perhaps most surprisingly, despite the huge differences in the lifestyles of the rich and the poor, we find the fewest research topic differences between people with annual household incomes over $100,000 USD relative to those making less than $20,000.
Tables illustrating the Top 20 topics by personality variable, demographic variable, and their differences are provided in the full report PDF file at the bottom of this page. A sample table, for Gender differences, is presented below.
First, there were differences with respect to which topics made the “Top 20” for each sub-sample. For example, the topic “anxiety” made introverts’ Top 20 list, but not extroverts’ (we listed any topic that made either top 20 list). Introverts also tend to want a bit more research on music and education than extroverts.
“Agreeable” people would like to see some more research on cancer, climate change, and children, along with depression and mental health (saving the world and the stress from it?) than those with a “tell it like it is” attitude.
Women want a bit more research on children and school, whereas men prefer more research on video games and sports.
Graduates with the highest levels of education want more research on “effects” showing causality, as opposed to mere correlations or polls, whereas those with less education prefer more research on cancer.
And perhaps most surprisingly, despite the huge differences in the lifestyles of the rich and the poor, we find the fewest research topic differences between people with annual household incomes over $100,000 USD relative to those making less than $20,000.
Tables illustrating the Top 20 topics by personality variable, demographic variable, and their differences are provided in the full report PDF file at the bottom of this page. A sample table, for Gender differences, is presented below.
References
Boroush, M. (2015). U.S. R&D Increased in 2013, Well Ahead of the Pace of Gross Domestic Product. National Science Foundation InfoBrief. NSF 15-330.
Evans, J. A. (2008). Electronic publication and the narrowing of science and scholarship. Science, 321(5887), 395-399.
Larivière, V., Gingras, Y. and Archambault, É. (2009), The decline in the concentration of citations, 1900–2007. Journal of the American Society for Information Science and Technology, 60: 858-862.
Methods Note:
To test for significant differences in the number of times each Top topic was listed between people of different personalities and demographics, we used a statistical test called a Chi-squared test. This test is commonly used to measure differences between two categorical variables, such as gender (e.g., Female = 1, Male = 0) and whether or not “video games” was listed as a topic (e.g., listed = 1, not listed = 0). For example, if men list video games far more frequently than women, the Chi-square statistic would be large and its corresponding “p-value” would be small. If the p-value is less than 0.05, we consider the difference statistically significant.
To measure the size of those differences, we calculated a statistic called “Cramer’s V,” which is a standardized effect size that makes the differences more comparable across groups (e.g., comparing a difference of 1% vs. 10% to 51% vs. 60%). As you’ll see in the tables below, most of the statistically significant differences have standardized effect sizes around 0.10, which is small (0.3 = medium, 0.5 = large). Yellow highlighting indicates a statistically significant, albeit small difference. Lighter yellow highlighting indicates a difference that was only borderline significant, but still worth noting.
Each table includes stats for the Top 20 topics from the overall sample of 1,000 people and any Top 20 topics from either sub-sample of usually 250 people. For example, the topic “people” will appear in every table because it was a Top 20 topic from the overall sample, but the topic “trump” will only appear when it’s a Top 20 topic from one of the sub-samples (e.g., top or bottom 25% of income earners).
Finally, a few words were combined in the analyses to ensure their differences were not merely due to wording (e.g., climate change vs. global warming). The following words were combined in the statistical tables and corresponding chi-square tests: climate change + global warming (listed as “climate change”), video game + video games (“video games”), politics + political (“politics”), and effects + effect (“effects”). However, the word clouds above each table do not combine these words to better represent the original topics as listed by participants.
For more details about this study, including the pre-registration, survey materials, and data, check out our page on the Open Science Framework.
Boroush, M. (2015). U.S. R&D Increased in 2013, Well Ahead of the Pace of Gross Domestic Product. National Science Foundation InfoBrief. NSF 15-330.
Evans, J. A. (2008). Electronic publication and the narrowing of science and scholarship. Science, 321(5887), 395-399.
Larivière, V., Gingras, Y. and Archambault, É. (2009), The decline in the concentration of citations, 1900–2007. Journal of the American Society for Information Science and Technology, 60: 858-862.
Methods Note:
To test for significant differences in the number of times each Top topic was listed between people of different personalities and demographics, we used a statistical test called a Chi-squared test. This test is commonly used to measure differences between two categorical variables, such as gender (e.g., Female = 1, Male = 0) and whether or not “video games” was listed as a topic (e.g., listed = 1, not listed = 0). For example, if men list video games far more frequently than women, the Chi-square statistic would be large and its corresponding “p-value” would be small. If the p-value is less than 0.05, we consider the difference statistically significant.
To measure the size of those differences, we calculated a statistic called “Cramer’s V,” which is a standardized effect size that makes the differences more comparable across groups (e.g., comparing a difference of 1% vs. 10% to 51% vs. 60%). As you’ll see in the tables below, most of the statistically significant differences have standardized effect sizes around 0.10, which is small (0.3 = medium, 0.5 = large). Yellow highlighting indicates a statistically significant, albeit small difference. Lighter yellow highlighting indicates a difference that was only borderline significant, but still worth noting.
Each table includes stats for the Top 20 topics from the overall sample of 1,000 people and any Top 20 topics from either sub-sample of usually 250 people. For example, the topic “people” will appear in every table because it was a Top 20 topic from the overall sample, but the topic “trump” will only appear when it’s a Top 20 topic from one of the sub-samples (e.g., top or bottom 25% of income earners).
Finally, a few words were combined in the analyses to ensure their differences were not merely due to wording (e.g., climate change vs. global warming). The following words were combined in the statistical tables and corresponding chi-square tests: climate change + global warming (listed as “climate change”), video game + video games (“video games”), politics + political (“politics”), and effects + effect (“effects”). However, the word clouds above each table do not combine these words to better represent the original topics as listed by participants.
For more details about this study, including the pre-registration, survey materials, and data, check out our page on the Open Science Framework.
|
Daniel BrownDaniel Brown is an award-winning researcher, social scientist, and founder of AB Labs. He holds a doctorate in Management from Harvard University where he was awarded the Wyss Award for Excellence in Research in 2019. He has conducted over 100 randomized controlled trials in social psychology.
|
Popular Experiments
COVID MasksDoes wearing a COVID mask affect how others think of you?
|
Video GamesAre video games more enjoyable than board games?
Does age or gender matter? |
Zero-Sum PoliticsDo Democrats or Republicans view society as win-lose?
|