Data studyBrain and memory · Version 2026.10
Which famous brain and intelligence findings held up?
A replication scorecard: 56 famous lab findings about memory, attention, self-control and decisions, each matched with its largest independent replication. What held up, how much the effects shrank, and whether science stopped citing the ones that failed.
By Leon Falkner, Data & Research Analyst · Reviewed by Aarav Vardhan, Editor-in-Chief · Published 27 October 2026 · 10 min read
About this study
- Report type
- Data study: original findings vs their replications
- Data
- 56 famous findings (originals 1935 to 2017), replications 1999 to 2024
- Covers
- 90,018 replication participants; originals cited 111,086 times in total
- Sources
- FReD (CC BY 4.0), replication reports, OpenAlex (CC0), Crossref
- Licence
- CC BY 4.0 (our table)
- Version
- 2026.10
- Sources
- 99
Short answer
Only 24 of 56 famous brain and intelligence findings (43%) held up when large, independent teams repeated them. We matched each finding, cited 404 to 18,624 times, with its biggest replication (1999 to 2024; 90,018 participants in total): 26 failed and 6 came back weaker. The typical replication effect was 62% smaller than the original. Data as of October 2026.
56 famous findings43% held upReplication crisisCC BY 4.0
43%of 56 famous findings held up
62%smaller: typical replication effect vs the original
+18%median change in citations after a failed replication
01 Key findings
- 43% held up (24 of 56); 6 came back weaker and 26 failed (46%).IQ Metrics Research replication scorecard
- Replications were typically 62% smaller: the median replication effect was 0.38 times the original (42 findings on a comparable scale).IQ Metrics Research replication scorecard
- Failed findings kept being cited. In the 3 years after a failed replication, citations of the original changed by a median +18%, against +8% for findings that held up.OpenAlex citations by year
- The most-cited failed finding is the claim that growth mindset predicts better grades, from Blackwell et al. (2007), cited 3,780 times.OpenAlex; Li & Bates (2020)
- Small studies failed more often. 50% of findings from originals with under 100 participants failed, against 32% from larger originals.IQ Metrics Research replication scorecard
02 Statistics to quote
43%
of 56 famous brain and intelligence findings held up in their largest independent replication.
IQ Metrics Research46%
failed outright: no effect, or the opposite effect.
IQ Metrics Research62%
smaller: the typical replication effect compared with the original.
IQ Metrics Research3,780
citations for the most-cited failed finding: growth mindset predicts better grades.
OpenAlex03 How many famous brain findings held up?
We took every famous finding about memory, learning, attention, reasoning, self-control, decisions and infant minds that has a large, independent replication with an effect size, and put the two side by side. “Famous” means the original paper has been cited at least 400 times. 56 findings qualify. 24 held up, 6 came back weaker and 26 failed.
Figure 1
How 56 famous findings fared when they were repeated
Outcome of the largest independent replication of each finding, by our stated rule.
26Failed46%
6Weaker11%
24Held up43%
Source: IQ Metrics Research replication scorecard [1].
Data table
| set | Failed | Weaker | Held up |
|---|---|---|---|
| Famous findings (Failed: no effect, or the opposite effect. Weaker: real but small (under r = .10) or under half the original. Held up: the rest.) | 26 | 6 | 24 |
Embed this chart
Paste this HTML into your page: it carries the numbers and the credit link to this figure.
Figure 2 shows every finding. The square is the original result and the dot is the replication; the lines are 95% confidence intervals. When the dot sits on the zero line, the replication found nothing. The findings that held up are mostly classic judgment effects such as anchoring and the “less is better” effect, which replicate at or above their original size.
Figure 2
Every famous finding: the original effect and its replication
Effect size as a correlation (r), with 95% confidence intervals. Square = the original study; dot = the largest independent replication, coloured by outcome. 49 of the 56 findings, grouped by topic.
Memory and learning1 held up, 2 failed
The "Google effect" on memory2011 → 20180.37→0.05Failed
Writing about worries boosts exam scores2011 → 20180.79→-0.10Failed
Retrieval practice beats concept mapping2011 → 20180.60→0.38Held up
Attention and thinking2 held up, 2 failed
Listening to Mozart boosts spatial reasoning (the Mozart effect)1993 → 1999–→0.03Failed
A nearby smartphone drains brainpower2017 → 20220.16→0.02Failed
Stroop effect1935 → 2016–→0.40Held up
People find time alone with their thoughts unpleasant2014 → 20180.67→0.59Held up
Reasoning, intelligence and mindset1 held up, 2 weaker, 6 failed
Growth mindset predicts better grades2007 → 20200.53→-0.05Failed
Reading literary fiction improves mind-reading2013 → 20180.27→-0.03Failed
Deep thinkers are swayed more by strong arguments1983 → 20160.41→0.01Failed
Hard-to-read fonts improve reasoning2007 → 20180.30→-0.01Failed
Unlikely events are thought of more abstractly2006 → 20200.52→0.02Failed
The curse of knowledge2007 → 20230.23→0.10Failed
Praising intelligence (not effort) makes children give up1998 → 20190.56→0.14Weaker
Low numeracy, bigger framing effect2006 → 20230.23→0.08Weaker
Analogies plus a stated principle help solve new problems1983 → 20230.22→0.28Held up
Self-control and willpower1 held up, 3 failed
Ego depletion (willpower runs out)1998 → 2004–→-0.09Failed
Mental fatigue lowers physical endurance2009 → 20190.17→-0.06Failed
Self-control tasks drain later self-control2007 → 20130.21→-0.15Failed
Preschool self-control (the marshmallow test) predicts later achievement1990 → 20180.57→0.30Held up
Judgment and decision-making14 held up, 2 weaker, 5 failed
Scarcity tires the mind for later tasks2012 → 20180.27→-0.01Failed
Deep thinkers adjust more from their own anchors2006 → 2024–→-0.05Failed
Emotion makes people ignore probability2001 → 20180.35→-0.04Failed
Choosing versus rejecting1993 → 20180.17→-0.07Failed
Scarcity changes how people judge value2015 → 20210.26→0.09Failed
Framing effect (the "Asian disease" problem)1981 → 20180.47→0.20Weaker
Availability heuristic1973 → 20160.38→0.04Weaker
False consensus effect1977 → 20180.37→0.43Held up
Norm theory: regret after unusual choices1986 → 20190.75→0.84Held up
Most drivers think they are above average1981 → 2023–→0.81Held up
Bias blind spot2002 → 20210.87→0.79Held up
Goals act as reference points1999 → 20240.62→0.44Held up
Outcome bias: good results make decisions look good1988 → 20230.10→0.36Held up
Anchoring in estimates1995 → 20140.42→0.74Held up
Feelings make people ignore quantity2004 → 20240.15→0.11Held up
Money illusion1997 → 20210.26→0.28Held up
Disjunction effect1992 → 20210.22→0.30Held up
Hindsight bias1977 → 2021–→0.20Held up
Frequencies do not remove the conjunction fallacy2001 → 20210.28→0.24Held up
The effort heuristic2004 → 20230.17→0.11Held up
Less is better1998 → 20180.33→0.36Held up
Priming and embodied cognition2 weaker, 4 failed
Elderly words make people walk slower1996 → 20120.47→-0.10Failed
Weight in your hands sways how you judge a job candidate2010 → 20180.27→0.06Failed
Thinking about distance weakens felt bonds to family2008 → 20150.30→0.05Failed
Heavy clipboards make things seem important2009 → 20160.28→0.01Failed
Spatial metaphors shape how we think about time2000 → 20160.30→0.13Weaker
Hostility priming2004 → 2018–→0.03Weaker
Babies and young children1 held up, 1 failed
Babies learn abstract rules at 7 months1999 → 20220.80→0.04Failed
Babies track what others believe2010 → 20180.45→0.59Held up
Effect size r (0 = no effect). No square: the original result is on a scale that cannot be put next to the replication. Not drawn (results in rating or percentage points, no single estimate, or effects measured across conditions or traits rather than people): Affect heuristic; People rate good traits as more like themselves; Better than average at easy things, worse at hard ones; Holding a pen in your teeth makes cartoons funnier (facial feedback); Thinking about professors improves trivia scores; Describing a face makes it harder to recognise (verbal overshadowing); Dual n-back training raises fluid intelligence (brain training).
Source: IQ Metrics Research replication scorecard, from FReD [1] and the replication reports listed under Sources; CIs computed as described in the method.
Data table
| finding | detail | group | original | original_ci_low | original_ci_high | original_n | replication | replication_ci_low | replication_ci_high | replication_n | outcome |
|---|---|---|---|---|---|---|---|---|---|---|---|
| The "Google effect" on memory | 2011 → 2018 | Memory and learning | 0.4 | 0.1 | 0.6 | 69 | 0.1 | -0.1 | 0.2 | 234 | Failed |
| Writing about worries boosts exam scores | 2011 → 2018 | Memory and learning | 0.8 | 0.5 | 0.9 | 20 | -0.1 | -0.3 | 0.1 | 79 | Failed |
| Retrieval practice beats concept mapping | 2011 → 2018 | Memory and learning | 0.6 | 0.4 | 0.8 | 40 | 0.4 | 0.1 | 0.6 | 49 | Held up |
| Listening to Mozart boosts spatial reasoning (the Mozart effect) | 1993 → 1999 | Attention and thinking | 36 | 0 | -0.2 | 0.2 | 125 | Failed | |||
| A nearby smartphone drains brainpower | 2017 → 2022 | Attention and thinking | 0.2 | 0 | 0.3 | 275 | 0 | -0.1 | 0.1 | 383 | Failed |
| Stroop effect | 1935 → 2016 | Attention and thinking | 100 | 0.4 | 0.4 | 0.4 | 3337 | Held up | |||
| People find time alone with their thoughts unpleasant | 2014 → 2018 | Attention and thinking | 0.7 | 0.4 | 0.8 | 30 | 0.6 | 0.3 | 0.8 | 39 | Held up |
| Growth mindset predicts better grades | 2007 → 2020 | Reasoning, intelligence and mindset | 0.5 | 0.5 | 0.6 | 373 | -0.1 | -0.1 | 0.1 | 586 | Failed |
| Reading literary fiction improves mind-reading | 2013 → 2018 | Reasoning, intelligence and mindset | 0.3 | 0.1 | 0.5 | 86 | -0 | -0.1 | 0 | 714 | Failed |
| Deep thinkers are swayed more by strong arguments | 1983 → 2016 | Reasoning, intelligence and mindset | 0.4 | 0.2 | 0.5 | 114 | 0 | -0 | 0 | 2365 | Failed |
| Hard-to-read fonts improve reasoning | 2007 → 2018 | Reasoning, intelligence and mindset | 0.3 | -0 | 0.5 | 41 | -0 | -0 | 0 | 2580 | Failed |
| Unlikely events are thought of more abstractly | 2006 → 2020 | Reasoning, intelligence and mindset | 0.5 | 0.3 | 0.7 | 34 | 0 | -0.2 | 0.2 | 120 | Failed |
| The curse of knowledge | 2007 → 2023 | Reasoning, intelligence and mindset | 0.2 | 0.1 | 0.4 | 155 | 0.1 | -0 | 0.2 | 225 | Failed |
| Praising intelligence (not effort) makes children give up | 1998 → 2019 | Reasoning, intelligence and mindset | 0.6 | 0.4 | 0.7 | 77 | 0.1 | 0 | 0.3 | 190 | Weaker |
| Low numeracy, bigger framing effect | 2006 → 2023 | Reasoning, intelligence and mindset | 0.2 | 0 | 0.4 | 100 | 0.1 | 0 | 0.1 | 860 | Weaker |
| Analogies plus a stated principle help solve new problems | 1983 → 2023 | Reasoning, intelligence and mindset | 0.2 | 0.1 | 0.4 | 143 | 0.3 | 0.1 | 0.4 | 175 | Held up |
| Ego depletion (willpower runs out) | 1998 → 2004 | Self-control and willpower | -0.1 | -0.3 | 0.1 | 76 | Failed | ||||
| Mental fatigue lowers physical endurance | 2009 → 2019 | Self-control and willpower | 0.2 | -0.4 | 0.6 | 16 | -0.1 | -0.4 | 0.3 | 30 | Failed |
| Self-control tasks drain later self-control | 2007 → 2013 | Self-control and willpower | 0.2 | 0 | 0.4 | 141 | -0.1 | -0.3 | -0 | 235 | Failed |
| Preschool self-control (the marshmallow test) predicts later achievement | 1990 → 2018 | Self-control and willpower | 0.6 | 0.3 | 0.8 | 35 | 0.3 | 0.2 | 0.4 | 918 | Held up |
| Scarcity tires the mind for later tasks | 2012 → 2018 | Judgment and decision-making | 0.3 | 0 | 0.5 | 56 | -0 | -0.1 | 0.1 | 619 | Failed |
| Deep thinkers adjust more from their own anchors | 2006 → 2024 | Judgment and decision-making | 81 | -0.1 | -0.2 | 0.1 | 303 | Failed | |||
| Emotion makes people ignore probability | 2001 → 2018 | Judgment and decision-making | 0.3 | 0 | 0.7 | 40 | -0 | -0.1 | -0 | 7218 | Failed |
| Choosing versus rejecting | 1993 → 2018 | Judgment and decision-making | 0.2 | -0 | 0.3 | 170 | -0.1 | -0.1 | -0 | 7901 | Failed |
| Scarcity changes how people judge value | 2015 → 2021 | Judgment and decision-making | 0.3 | 0 | 0.5 | 73 | 0.1 | -0 | 0.2 | 207 | Failed |
| Framing effect (the "Asian disease" problem) | 1981 → 2018 | Judgment and decision-making | 0.5 | 0.3 | 0.6 | 181 | 0.2 | 0.2 | 0.2 | 7228 | Weaker |
| Availability heuristic | 1973 → 2016 | Judgment and decision-making | 0.4 | 0.2 | 0.5 | 152 | 0 | 0 | 0.1 | 3088 | Weaker |
| False consensus effect | 1977 → 2018 | Judgment and decision-making | 0.4 | 0.1 | 0.7 | 80 | 0.4 | 0.4 | 0.4 | 7827 | Held up |
| Norm theory: regret after unusual choices | 1986 → 2019 | Judgment and decision-making | 0.8 | 0.7 | 0.8 | 138 | 0.8 | 0.8 | 0.9 | 342 | Held up |
| Most drivers think they are above average | 1981 → 2023 | Judgment and decision-making | 86 | 0.8 | 0.8 | 0.8 | 1203 | Held up | |||
| Bias blind spot | 2002 → 2021 | Judgment and decision-making | 0.9 | 0.7 | 0.9 | 24 | 0.8 | 0.8 | 0.8 | 621 | Held up |
| Goals act as reference points | 1999 → 2024 | Judgment and decision-making | 0.6 | 0.5 | 0.7 | 45 | 0.4 | 0.4 | 0.5 | 377 | Held up |
| Outcome bias: good results make decisions look good | 1988 → 2023 | Judgment and decision-making | 0.1 | -0.4 | 0.5 | 20 | 0.4 | 0.3 | 0.4 | 692 | Held up |
| Anchoring in estimates | 1995 → 2014 | Judgment and decision-making | 0.4 | 0.2 | 0.6 | 104 | 0.7 | 0.7 | 0.8 | 5627 | Held up |
| Feelings make people ignore quantity | 2004 → 2024 | Judgment and decision-making | 0.1 | 0 | 0.3 | 331 | 0.1 | 0 | 0.2 | 1041 | Held up |
| Money illusion | 1997 → 2021 | Judgment and decision-making | 0.3 | 0.2 | 0.4 | 358 | 0.3 | 0.2 | 0.4 | 604 | Held up |
| Disjunction effect | 1992 → 2021 | Judgment and decision-making | 0.2 | 0.1 | 0.3 | 66 | 0.3 | 0.2 | 0.4 | 445 | Held up |
| Hindsight bias | 1977 → 2021 | Judgment and decision-making | 184 | 0.2 | 0.1 | 0.3 | 608 | Held up | |||
| Frequencies do not remove the conjunction fallacy | 2001 → 2021 | Judgment and decision-making | 0.3 | 0.2 | 0.4 | 108 | 0.2 | 0.2 | 0.3 | 1032 | Held up |
| The effort heuristic | 2004 → 2023 | Judgment and decision-making | 0.2 | 0 | 0.3 | 138 | 0.1 | 0 | 0.2 | 700 | Held up |
| Less is better | 1998 → 2018 | Judgment and decision-making | 0.3 | 0.1 | 0.5 | 83 | 0.4 | 0.3 | 0.4 | 7646 | Held up |
| Elderly words make people walk slower | 1996 → 2012 | Priming and embodied cognition | 0.5 | 0.1 | 0.7 | 30 | -0.1 | -0.3 | 0.1 | 120 | Failed |
| Weight in your hands sways how you judge a job candidate | 2010 → 2018 | Priming and embodied cognition | 0.3 | 0 | 0.5 | 54 | 0.1 | -0 | 0.1 | 599 | Failed |
| Thinking about distance weakens felt bonds to family | 2008 → 2015 | Priming and embodied cognition | 0.3 | 0.1 | 0.5 | 84 | 0.1 | -0.1 | 0.2 | 125 | Failed |
| Heavy clipboards make things seem important | 2009 → 2016 | Priming and embodied cognition | 0.3 | 0 | 0.5 | 51 | 0 | -0 | 0.1 | 2285 | Failed |
| Spatial metaphors shape how we think about time | 2000 → 2016 | Priming and embodied cognition | 0.3 | 0 | 0.5 | 56 | 0.1 | 0.1 | 0.2 | 1335 | Weaker |
| Hostility priming | 2004 → 2018 | Priming and embodied cognition | 96 | 0 | 0 | 0.1 | 5610 | Weaker | |||
| Babies learn abstract rules at 7 months | 1999 → 2022 | Babies and young children | 0.8 | 0.5 | 0.9 | 16 | 0 | -0.2 | 0.2 | 96 | Failed |
| Babies track what others believe | 2010 → 2018 | Babies and young children | 0.5 | 0.1 | 0.7 | 24 | 0.6 | 0.4 | 0.7 | 95 | Held up |
Embed this chart
Paste this HTML into your page: it carries the numbers and the credit link to this figure.
04 How much smaller were the replication effects?
Across the 42 findings whose original and replication effects are on the same scale, the median replication effect was 0.38 times the original: 62% smaller. Even many findings that “worked” came back smaller. The 15 most-cited findings show the pattern.
Figure 3
The 15 most-cited findings: claimed effect vs replicated effect
Effect size r in the original study and in its largest independent replication. Most-cited first.
Original studyReplication
Framing effect (the "Asian disease" problem)0.200.48
Availability heuristic0.040.38
Growth mindset predicts better grades-0.050.53
False consensus effect0.370.43
Norm theory: regret after unusual choices0.750.84
Analogies plus a stated principle help solve new problems0.220.28
Elderly words make people walk slower-0.100.47
Spatial metaphors shape how we think about time0.130.30
The "Google effect" on memory0.050.37
Praising intelligence (not effort) makes children give up0.140.56
Reading literary fiction improves mind-reading-0.030.27
Scarcity tires the mind for later tasks-0.010.27
Mental fatigue lowers physical endurance-0.060.17
Bias blind spot0.790.87
Babies learn abstract rules at 7 months0.040.80
Effect size r. Values below −0.20 are drawn at −0.20.
Source: IQ Metrics Research replication scorecard; citations from OpenAlex [2].
Data table
| finding | Original study | Replication |
|---|---|---|
| Framing effect (the "Asian disease" problem) | 0.5 | 0.2 |
| Availability heuristic | 0.4 | 0 |
| Growth mindset predicts better grades | 0.5 | -0.1 |
| False consensus effect | 0.4 | 0.4 |
| Norm theory: regret after unusual choices | 0.8 | 0.8 |
| Analogies plus a stated principle help solve new problems | 0.2 | 0.3 |
| Elderly words make people walk slower | 0.5 | -0.1 |
| Spatial metaphors shape how we think about time | 0.3 | 0.1 |
| The "Google effect" on memory | 0.4 | 0.1 |
| Praising intelligence (not effort) makes children give up | 0.6 | 0.1 |
| Reading literary fiction improves mind-reading | 0.3 | -0 |
| Scarcity tires the mind for later tasks | 0.3 | -0 |
| Mental fatigue lowers physical endurance | 0.2 | -0.1 |
| Bias blind spot | 0.9 | 0.8 |
| Babies learn abstract rules at 7 months | 0.8 | 0 |
Embed this chart
Paste this HTML into your page: it carries the numbers and the credit link to this figure.
Size alone does not decide how much survives: every group has findings that vanished and findings that kept most of their effect. The median share left was 0.53 for originals with under 50 people (15 findings), 0.38 for originals with 50 to 199 (26 findings), 0.73 for originals with 200 or more (5 findings).
Figure 4
How much of the original effect was left, by size of the original study
Replication effect divided by the original effect (1 = the same size, 0 = nothing left, below 0 = the opposite direction). One dot per finding.
Original under 50 people15 findings0.53
50 to 199 people26 findings0.38
200 people or more5 findings0.73
Ratios are clipped to the range −1 to 1.6 for drawing; the CSV has the exact values.
Source: IQ Metrics Research replication scorecard [1].
Data table
| original sample | item | value |
|---|---|---|
| Original under 50 people | Elderly words make people walk slower | -0.2 |
| Original under 50 people | Bias blind spot | 0.9 |
| Original under 50 people | Babies learn abstract rules at 7 months | 0.1 |
| Original under 50 people | Retrieval practice beats concept mapping | 0.6 |
| Original under 50 people | Goals act as reference points | 0.7 |
| Original under 50 people | Babies track what others believe | 1.3 |
| Original under 50 people | Hard-to-read fonts improve reasoning | -0.1 |
| Original under 50 people | Emotion makes people ignore probability | -0.1 |
| Original under 50 people | People find time alone with their thoughts unpleasant | 0.9 |
| Original under 50 people | Unlikely events are thought of more abstractly | 0 |
| Original under 50 people | Writing about worries boosts exam scores | -0.1 |
| Original under 50 people | Mental fatigue lowers physical endurance | -0.4 |
| Original under 50 people | Outcome bias: good results make decisions look good | 1.6 |
| Original under 50 people | Better than average at easy things, worse at hard ones | 0.9 |
| Original under 50 people | Preschool self-control (the marshmallow test) predicts later achievement | 0.5 |
| 50 to 199 people | Framing effect (the "Asian disease" problem) | 0.4 |
| 50 to 199 people | Availability heuristic | 0.1 |
| 50 to 199 people | False consensus effect | 1.2 |
| 50 to 199 people | Norm theory: regret after unusual choices | 1.1 |
| 50 to 199 people | Analogies plus a stated principle help solve new problems | 1.3 |
| 50 to 199 people | Spatial metaphors shape how we think about time | 0.4 |
| 50 to 199 people | The "Google effect" on memory | 0.1 |
| 50 to 199 people | Reading literary fiction improves mind-reading | -0.1 |
| 50 to 199 people | Scarcity tires the mind for later tasks | -0.1 |
| 50 to 199 people | Low numeracy, bigger framing effect | 0.3 |
| 50 to 199 people | Anchoring in estimates | 1.6 |
| 50 to 199 people | Self-control tasks drain later self-control | -0.7 |
| 50 to 199 people | Disjunction effect | 1.4 |
| 50 to 199 people | Weight in your hands sways how you judge a job candidate | 0.2 |
| 50 to 199 people | Choosing versus rejecting | -0.4 |
| 50 to 199 people | The curse of knowledge | 0.4 |
| 50 to 199 people | The effort heuristic | 0.7 |
| 50 to 199 people | Heavy clipboards make things seem important | 0.1 |
| 50 to 199 people | Less is better | 1.1 |
| 50 to 199 people | Frequencies do not remove the conjunction fallacy | 0.8 |
| 50 to 199 people | Scarcity changes how people judge value | 0.3 |
| 50 to 199 people | Thinking about distance weakens felt bonds to family | 0.2 |
| 50 to 199 people | Deep thinkers are swayed more by strong arguments | 0 |
| 50 to 199 people | People rate good traits as more like themselves | 0.8 |
| 50 to 199 people | Describing a face makes it harder to recognise (verbal overshadowing) | 0.7 |
| 50 to 199 people | Praising intelligence (not effort) makes children give up | 0.2 |
| 200 people or more | Growth mindset predicts better grades | -0.1 |
| 200 people or more | Affect heuristic | 1.2 |
| 200 people or more | A nearby smartphone drains brainpower | 0.1 |
| 200 people or more | Feelings make people ignore quantity | 0.7 |
| 200 people or more | Money illusion | 1.1 |
Embed this chart
Paste this HTML into your page: it carries the numbers and the credit link to this figure.
05 Which kinds of findings held up, and which failed?
Judgment and decision-making effects held up most often: 17 of 24. At the other end, none of the 8 priming and embodied cognition findings held up, and 1 of the 10 reasoning, intelligence and mindset findings.
Figure 5
Which kinds of findings held up
One dot per finding, by topic and outcome.
Held upWeakerFailedMemory and learning4 findings22Attention and thinking4 findings22Reasoning, intelligence and mindset10 findings127Self-control and willpower4 findings13Judgment and decision-making24 findings1725Priming and embodied cognition8 findings26Babies and young children2 findings11
Source: IQ Metrics Research replication scorecard [1].
Data table
| id | topic | outcome | name |
|---|---|---|---|
| stroop effect | Attention and thinking | Held up | Stroop effect |
| framing effect (the "asian disease" problem) | Judgment and decision-making | Weaker | Framing effect (the "Asian disease" problem) |
| availability heuristic | Judgment and decision-making | Weaker | Availability heuristic |
| growth mindset predicts better grades | Reasoning, intelligence and mindset | Failed | Growth mindset predicts better grades |
| false consensus effect | Judgment and decision-making | Held up | False consensus effect |
| norm theory: regret after unusual choices | Judgment and decision-making | Held up | Norm theory: regret after unusual choices |
| analogies plus a stated principle help solve new problems | Reasoning, intelligence and mindset | Held up | Analogies plus a stated principle help solve new problems |
| affect heuristic | Judgment and decision-making | Held up | Affect heuristic |
| elderly words make people walk slower | Priming and embodied cognition | Failed | Elderly words make people walk slower |
| most drivers think they are above average | Judgment and decision-making | Held up | Most drivers think they are above average |
| spatial metaphors shape how we think about time | Priming and embodied cognition | Weaker | Spatial metaphors shape how we think about time |
| the "google effect" on memory | Memory and learning | Failed | The "Google effect" on memory |
| ego depletion (willpower runs out) | Self-control and willpower | Failed | Ego depletion (willpower runs out) |
| reading literary fiction improves mind-reading | Reasoning, intelligence and mindset | Failed | Reading literary fiction improves mind-reading |
| scarcity tires the mind for later tasks | Judgment and decision-making | Failed | Scarcity tires the mind for later tasks |
| bias blind spot | Judgment and decision-making | Held up | Bias blind spot |
| babies learn abstract rules at 7 months | Babies and young children | Failed | Babies learn abstract rules at 7 months |
| low numeracy, bigger framing effect | Reasoning, intelligence and mindset | Weaker | Low numeracy, bigger framing effect |
| retrieval practice beats concept mapping | Memory and learning | Held up | Retrieval practice beats concept mapping |
| goals act as reference points | Judgment and decision-making | Held up | Goals act as reference points |
| babies track what others believe | Babies and young children | Held up | Babies track what others believe |
| deep thinkers adjust more from their own anchors | Judgment and decision-making | Failed | Deep thinkers adjust more from their own anchors |
| hard-to-read fonts improve reasoning | Reasoning, intelligence and mindset | Failed | Hard-to-read fonts improve reasoning |
| emotion makes people ignore probability | Judgment and decision-making | Failed | Emotion makes people ignore probability |
| a nearby smartphone drains brainpower | Attention and thinking | Failed | A nearby smartphone drains brainpower |
| anchoring in estimates | Judgment and decision-making | Held up | Anchoring in estimates |
| feelings make people ignore quantity | Judgment and decision-making | Held up | Feelings make people ignore quantity |
| self-control tasks drain later self-control | Self-control and willpower | Failed | Self-control tasks drain later self-control |
| disjunction effect | Judgment and decision-making | Held up | Disjunction effect |
| weight in your hands sways how you judge a job candidate | Priming and embodied cognition | Failed | Weight in your hands sways how you judge a job candidate |
| choosing versus rejecting | Judgment and decision-making | Failed | Choosing versus rejecting |
| people find time alone with their thoughts unpleasant | Attention and thinking | Held up | People find time alone with their thoughts unpleasant |
| unlikely events are thought of more abstractly | Reasoning, intelligence and mindset | Failed | Unlikely events are thought of more abstractly |
| the curse of knowledge | Reasoning, intelligence and mindset | Failed | The curse of knowledge |
| the effort heuristic | Judgment and decision-making | Held up | The effort heuristic |
| writing about worries boosts exam scores | Memory and learning | Failed | Writing about worries boosts exam scores |
| heavy clipboards make things seem important | Priming and embodied cognition | Failed | Heavy clipboards make things seem important |
| less is better | Judgment and decision-making | Held up | Less is better |
| mental fatigue lowers physical endurance | Self-control and willpower | Failed | Mental fatigue lowers physical endurance |
| money illusion | Judgment and decision-making | Held up | Money illusion |
| frequencies do not remove the conjunction fallacy | Judgment and decision-making | Held up | Frequencies do not remove the conjunction fallacy |
| scarcity changes how people judge value | Judgment and decision-making | Failed | Scarcity changes how people judge value |
| thinking about distance weakens felt bonds to family | Priming and embodied cognition | Failed | Thinking about distance weakens felt bonds to family |
| deep thinkers are swayed more by strong arguments | Reasoning, intelligence and mindset | Failed | Deep thinkers are swayed more by strong arguments |
| people rate good traits as more like themselves | Judgment and decision-making | Held up | People rate good traits as more like themselves |
| outcome bias: good results make decisions look good | Judgment and decision-making | Held up | Outcome bias: good results make decisions look good |
| better than average at easy things, worse at hard ones | Judgment and decision-making | Held up | Better than average at easy things, worse at hard ones |
| hindsight bias | Judgment and decision-making | Held up | Hindsight bias |
| hostility priming | Priming and embodied cognition | Weaker | Hostility priming |
| facial-feedback | Priming and embodied cognition | Failed | Holding a pen in your teeth makes cartoons funnier (facial feedback) |
| professor-priming | Priming and embodied cognition | Failed | Thinking about professors improves trivia scores |
| verbal-overshadowing | Memory and learning | Held up | Describing a face makes it harder to recognise (verbal overshadowing) |
| marshmallow-test | Self-control and willpower | Held up | Preschool self-control (the marshmallow test) predicts later achievement |
| praise-for-intelligence | Reasoning, intelligence and mindset | Weaker | Praising intelligence (not effort) makes children give up |
| dual-n-back-training | Reasoning, intelligence and mindset | Failed | Dual n-back training raises fluid intelligence (brain training) |
| mozart-effect | Attention and thinking | Failed | Listening to Mozart boosts spatial reasoning (the Mozart effect) |
Embed this chart
Paste this HTML into your page: it carries the numbers and the credit link to this figure.
06 Did small studies fail more often?
50% of findings from original studies with fewer than 100 participants failed (34 findings), against 32% from larger original studies (19 findings). Big original effects were not the warning sign: 37% of originals reporting an effect of r = .30 or more failed, against 58% of those below .30.
Figure 6
Small original studies failed more often
Each bubble is one original study: its sample size and effect size. Bubble area = citations. Colour = what happened in the replication.
Held upWeakerFailed
Original effect size (r)
Availability heuristicNorm theory: regret after unusual choicesBias blind spotGoals act as reference pointsBabies track what others believeOutcome bias: good results make decisions look goodDeep thinkers are swayed more by strong argumentsFeelings make people ignore quantityMoney illusionDisjunction effectPeople find time alone with their thoughts unpleasantThe effort heuristicWriting about worries boosts exam scoresLess is better
Participants in the original study (log scale)
A link, not proof. Each dot is one original study; the replication outcome is its colour.
Source: IQ Metrics Research replication scorecard; citations from OpenAlex [2].
Data table
| finding | original_n | original_r | group | citations |
|---|---|---|---|---|
| Framing effect (the "Asian disease" problem) | 181 | 0.5 | weaker | 17834 |
| Availability heuristic | 152 | 0.4 | weaker | 9977 |
| Growth mindset predicts better grades | 373 | 0.5 | failed | 3780 |
| False consensus effect | 80 | 0.4 | held | 3163 |
| Norm theory: regret after unusual choices | 138 | 0.8 | held | 3027 |
| Analogies plus a stated principle help solve new problems | 143 | 0.2 | held | 2933 |
| Elderly words make people walk slower | 30 | 0.5 | failed | 2736 |
| Spatial metaphors shape how we think about time | 56 | 0.3 | weaker | 1841 |
| The "Google effect" on memory | 69 | 0.4 | failed | 1744 |
| Reading literary fiction improves mind-reading | 86 | 0.3 | failed | 1641 |
| Scarcity tires the mind for later tasks | 56 | 0.3 | failed | 1562 |
| Bias blind spot | 24 | 0.9 | held | 1285 |
| Babies learn abstract rules at 7 months | 16 | 0.8 | failed | 1246 |
| Low numeracy, bigger framing effect | 100 | 0.2 | weaker | 1178 |
| Retrieval practice beats concept mapping | 40 | 0.6 | held | 1014 |
| Goals act as reference points | 45 | 0.6 | held | 1008 |
| Babies track what others believe | 24 | 0.5 | held | 923 |
| Hard-to-read fonts improve reasoning | 41 | 0.3 | failed | 844 |
| Emotion makes people ignore probability | 40 | 0.3 | failed | 825 |
| A nearby smartphone drains brainpower | 275 | 0.2 | failed | 783 |
| Anchoring in estimates | 104 | 0.4 | held | 753 |
| Feelings make people ignore quantity | 331 | 0.1 | held | 654 |
| Self-control tasks drain later self-control | 141 | 0.2 | failed | 643 |
| Disjunction effect | 66 | 0.2 | held | 581 |
| Weight in your hands sways how you judge a job candidate | 54 | 0.3 | failed | 575 |
| Choosing versus rejecting | 170 | 0.2 | failed | 572 |
| People find time alone with their thoughts unpleasant | 30 | 0.7 | held | 536 |
| Unlikely events are thought of more abstractly | 34 | 0.5 | failed | 468 |
| The curse of knowledge | 155 | 0.2 | failed | 451 |
| The effort heuristic | 138 | 0.2 | held | 438 |
| Writing about worries boosts exam scores | 20 | 0.8 | failed | 433 |
| Heavy clipboards make things seem important | 51 | 0.3 | failed | 404 |
| Less is better | 83 | 0.3 | held | 404 |
| Mental fatigue lowers physical endurance | 16 | 0.2 | failed | 1450 |
| Money illusion | 358 | 0.3 | held | 616 |
| Frequencies do not remove the conjunction fallacy | 108 | 0.3 | held | 448 |
| Scarcity changes how people judge value | 73 | 0.3 | failed | 432 |
| Thinking about distance weakens felt bonds to family | 84 | 0.3 | failed | 459 |
| Deep thinkers are swayed more by strong arguments | 114 | 0.4 | failed | 849 |
| Outcome bias: good results make decisions look good | 20 | 0.1 | held | 872 |
| Preschool self-control (the marshmallow test) predicts later achievement | 35 | 0.6 | held | 1246 |
| Praising intelligence (not effort) makes children give up | 77 | 0.6 | weaker | 1703 |
| Dual n-back training raises fluid intelligence (brain training) | 69 | 0.3 | failed | 2183 |
Embed this chart
Paste this HTML into your page: it carries the numbers and the credit link to this figure.
07 Did scientists stop citing findings that failed?
No. For the 24 failed findings whose replication is old enough to judge, citations in the 3 years after the replication changed by a median +18% compared with the 3 years before. Findings that held up changed by +8% (18 findings). A failed replication did not stop a finding from being cited.
Figure 7
Citations before and after the replication came out
Citations per year, median across findings, indexed to 100 in the year before the replication (0 = the year it was published). Failed: 19 findings; held up: 10 (only findings with five full years after the replication).
Failed findingsFindings that held up
9395
Source: IQ Metrics Research replication scorecard; citations by year from OpenAlex [2].
Data table
| x | Failed findings | Findings that held up |
|---|---|---|
| -5 | 72 | 103 |
| -4 | 87 | 103 |
| -3 | 92 | 102 |
| -2 | 96 | 104 |
| -1 | 100 | 100 |
| Replication year | 117 | 138 |
| +1 | 100 | 120 |
| +2 | 107 | 120 |
| +3 | 121 | 119 |
| +4 | 97 | 114 |
| +5 | 93 | 95 |
Embed this chart
Paste this HTML into your page: it carries the numbers and the credit link to this figure.
Figure 8
The most-cited failed findings: citations per year
Dashed line = the year the replication was published. Each panel has its own scale.
Growth mindset predicts better gradesBlackwell et al. (2007); replication 2020
Replication 2020
20002025
Before 282 a yearAfter 280 a year
Elderly words make people walk slowerBargh et al. (1996); replication 2012
Replication 2012
20002025
Before 108 a yearAfter 136 a year
Dual n-back training raises fluid intelligence (brain training)Jaeggi et al. (2008); replication 2013
Replication 2013
20002025
Before 107 a yearAfter 149 a year
Holding a pen in your teeth makes cartoons funnier (facial feedback)Strack et al. (1988); replication 2016
Replication 2016
20002025
Before 93 a yearAfter 128 a year
The "Google effect" on memorySparrow et al. (2011); replication 2018
Replication 2018
20002025
Before 75 a yearAfter 108 a year
Ego depletion (willpower runs out)Muraven et al. (1998); replication 2004
Replication 2004
20002025
Before 21 a yearAfter 40 a year
Before = average of the 3 years before the replication; after = the 3 years after it.
Source: OpenAlex citations by year [2]; replication years from the reports.
Data table
| finding | x | value |
|---|---|---|
| Growth mindset predicts better grades | 2000 | 2 |
| Growth mindset predicts better grades | 2001 | 0 |
| Growth mindset predicts better grades | 2002 | 0 |
| Growth mindset predicts better grades | 2003 | 1 |
| Growth mindset predicts better grades | 2004 | 1 |
| Growth mindset predicts better grades | 2005 | 1 |
| Growth mindset predicts better grades | 2006 | 2 |
| Growth mindset predicts better grades | 2007 | 11 |
| Growth mindset predicts better grades | 2008 | 29 |
| Growth mindset predicts better grades | 2009 | 42 |
| Growth mindset predicts better grades | 2010 | 45 |
| Growth mindset predicts better grades | 2011 | 78 |
| Growth mindset predicts better grades | 2012 | 107 |
| Growth mindset predicts better grades | 2013 | 130 |
| Growth mindset predicts better grades | 2014 | 169 |
| Growth mindset predicts better grades | 2015 | 188 |
| Growth mindset predicts better grades | 2016 | 242 |
| Growth mindset predicts better grades | 2017 | 247 |
| Growth mindset predicts better grades | 2018 | 257 |
| Growth mindset predicts better grades | 2019 | 341 |
| Growth mindset predicts better grades | 2020 | 358 |
| Growth mindset predicts better grades | 2021 | 324 |
| Growth mindset predicts better grades | 2022 | 264 |
| Growth mindset predicts better grades | 2023 | 253 |
| Growth mindset predicts better grades | 2024 | 256 |
| Growth mindset predicts better grades | 2025 | 237 |
| Elderly words make people walk slower | 2000 | 28 |
| Elderly words make people walk slower | 2001 | 38 |
| Elderly words make people walk slower | 2002 | 64 |
| Elderly words make people walk slower | 2003 | 70 |
| Elderly words make people walk slower | 2004 | 59 |
| Elderly words make people walk slower | 2005 | 77 |
| Elderly words make people walk slower | 2006 | 88 |
| Elderly words make people walk slower | 2007 | 75 |
| Elderly words make people walk slower | 2008 | 90 |
| Elderly words make people walk slower | 2009 | 102 |
| Elderly words make people walk slower | 2010 | 118 |
| Elderly words make people walk slower | 2011 | 104 |
| Elderly words make people walk slower | 2012 | 132 |
| Elderly words make people walk slower | 2013 | 137 |
| Elderly words make people walk slower | 2014 | 134 |
| Elderly words make people walk slower | 2015 | 138 |
| Elderly words make people walk slower | 2016 | 146 |
| Elderly words make people walk slower | 2017 | 132 |
| Elderly words make people walk slower | 2018 | 122 |
| Elderly words make people walk slower | 2019 | 153 |
| Elderly words make people walk slower | 2020 | 172 |
| Elderly words make people walk slower | 2021 | 119 |
| Elderly words make people walk slower | 2022 | 92 |
| Elderly words make people walk slower | 2023 | 86 |
| Elderly words make people walk slower | 2024 | 81 |
| Elderly words make people walk slower | 2025 | 85 |
| Dual n-back training raises fluid intelligence (brain training) | 2000 | 0 |
| Dual n-back training raises fluid intelligence (brain training) | 2001 | 0 |
| Dual n-back training raises fluid intelligence (brain training) | 2002 | 0 |
| Dual n-back training raises fluid intelligence (brain training) | 2003 | 0 |
| Dual n-back training raises fluid intelligence (brain training) | 2004 | 1 |
| Dual n-back training raises fluid intelligence (brain training) | 2005 | 0 |
| Dual n-back training raises fluid intelligence (brain training) | 2006 | 0 |
| Dual n-back training raises fluid intelligence (brain training) | 2007 | 0 |
| Dual n-back training raises fluid intelligence (brain training) | 2008 | 17 |
| Dual n-back training raises fluid intelligence (brain training) | 2009 | 39 |
| Dual n-back training raises fluid intelligence (brain training) | 2010 | 72 |
| Dual n-back training raises fluid intelligence (brain training) | 2011 | 115 |
| Dual n-back training raises fluid intelligence (brain training) | 2012 | 134 |
| Dual n-back training raises fluid intelligence (brain training) | 2013 | 158 |
| Dual n-back training raises fluid intelligence (brain training) | 2014 | 145 |
| Dual n-back training raises fluid intelligence (brain training) | 2015 | 158 |
| Dual n-back training raises fluid intelligence (brain training) | 2016 | 144 |
| Dual n-back training raises fluid intelligence (brain training) | 2017 | 159 |
| Dual n-back training raises fluid intelligence (brain training) | 2018 | 138 |
| Dual n-back training raises fluid intelligence (brain training) | 2019 | 153 |
| Dual n-back training raises fluid intelligence (brain training) | 2020 | 194 |
| Dual n-back training raises fluid intelligence (brain training) | 2021 | 126 |
| Dual n-back training raises fluid intelligence (brain training) | 2022 | 97 |
| Dual n-back training raises fluid intelligence (brain training) | 2023 | 85 |
| Dual n-back training raises fluid intelligence (brain training) | 2024 | 86 |
| Dual n-back training raises fluid intelligence (brain training) | 2025 | 93 |
| Holding a pen in your teeth makes cartoons funnier (facial feedback) | 2000 | 18 |
| Holding a pen in your teeth makes cartoons funnier (facial feedback) | 2001 | 15 |
First 80 of 156 rows; the Data button gives all of them.
Embed this chart
Paste this HTML into your page: it carries the numbers and the credit link to this figure.
Figure 9
From famous, to tested, to still cited
Each finding flows from its replication outcome to what happened to its citations in the 3 years after the replication, compared with the 3 years before.
Famous findingsFamous 56
Held upHeld up 24WeakerWeaker 6FailedFailed 26
Cited as much or more afterMore 33Cited less afterLess 14Too recent to tellToo recent 9
Source: IQ Metrics Research replication scorecard; citations from OpenAlex [2].
Data table
| from | to | from |
|---|---|---|
| Famous findings | Held up | 24 |
| Famous findings | Weaker | 6 |
| Famous findings | Failed | 26 |
| Held up | Cited as much or more after | 11 |
| Held up | Cited less after | 7 |
| Held up | Too recent to tell | 6 |
| Weaker | Cited as much or more after | 4 |
| Weaker | Cited less after | 1 |
| Weaker | Too recent to tell | 1 |
| Failed | Cited as much or more after | 18 |
| Failed | Cited less after | 6 |
| Failed | Too recent to tell | 2 |
Embed this chart
Paste this HTML into your page: it carries the numbers and the credit link to this figure.
Citations are not endorsements
A paper can be cited to criticise it. Counting citations shows that a finding is still part of the conversation, not that the people citing it still believe it.
08 Were any of these papers retracted or corrected?
None of the 56 original papers has been retracted or corrected. We checked every original paper’s DOI against Crossref’s retraction, correction and expression-of-concern notices [3] and OpenAlex’s retraction flag [2]. A failed replication is not misconduct: most of these studies were run honestly, with methods that were normal at the time.
09 Which famous claims are not on the scorecard, and why?
- Babies prefer helpers (Hamlin, Wynn and Bloom, 2007). The ManyBabies 4 project tested 567 infants in 37 labs: 49% of the 302 who saw the original puppet show chose the helper, no better than chance [4]. It is not scored because the original first author was part of the replication team, and our rule needs an independent team.
- Facial feedback, again. After the failed 2016 replication on the scorecard, the Many Smiles project (3,878 people in 19 countries) reported that its pen-in-mouth results were less conclusive [5]. We score the replication of the original pen study.
- Choice overload (too many jams) and the bilingual advantage. Both are famous, but neither has a large direct replication with an effect size in FReD or a multi-lab report, so they are not scored. Our page on bilingualism and IQ covers what the larger studies found.
- Left out because we could not read the replication report in full: Two-year-olds understand false beliefs; Reward cues boost memory; Babies prefer helpers to hinderers (2011 study); Holding people accountable reduces sunk-cost escalation. Every scored number was checked in a paper or report.
10 Where can I read more about these findings?
- Music and IQ: does the Mozart effect raise IQ? and music lessons and IQ.
- Brain training: do brain-training apps work? and what the dual n-back replications show.
- Self-control in children: the marshmallow test.
- Mindsets: growth mindset and intelligence. Languages: bilingualism and IQ.
- Myths about the brain that teachers believe: neuromyths by country. Viral brain numbers checked: brain statistics fact-check.
- The puzzle used in the “hard-to-read font” study: the world’s shortest IQ test.
11 Every finding in one table
Sortable: click a column heading. The CSV adds the exact statistic of each study, both DOIs, the confidence intervals, the replication team’s own verdict and citations per year.
Table 1
All 56 findings: original vs replication
| Stroop effect | Attention and thinking | Stroop (1935), n = 100 | Ebersole et al. (2016), n = 3,337 | – | r = 0.40 [0.39, 0.42] | Held up | 18,624 |
| Framing effect (the “Asian disease” problem) | Judgment and decision-making | Tversky & Kahneman (1981), n = 181 | Klein et al. (2018), n = 7,228 | r = 0.48 | r = 0.20 [0.17, 0.22] | Weaker | 17,834 |
| Availability heuristic | Judgment and decision-making | Tversky & Kahneman (1973), n = 152 | Ebersole et al. (2016), n = 3,088 | r = 0.38 | r = 0.04 [0.01, 0.08] | Weaker | 9,977 |
| Growth mindset predicts better grades | Reasoning, intelligence and mindset | Blackwell et al. (2007), n = 373 | Li & Bates (2020), n = 586 | r = 0.53 | r = -0.05 [-0.14, 0.05] | Failed | 3,780 |
| False consensus effect | Judgment and decision-making | Ross et al. (1977), n = 80 | Klein et al. (2018), n = 7,827 | r = 0.37 | r = 0.43 [0.41, 0.45] | Held up | 3,163 |
| Norm theory: regret after unusual choices | Judgment and decision-making | Kahneman & Miller (1986), n = 138 | Kutscher & Feldman (2019), n = 342 | r = 0.75 | r = 0.84 [0.81, 0.87] | Held up | 3,027 |
| Analogies plus a stated principle help solve new problems | Reasoning, intelligence and mindset | Gick & Holyoak (1983), n = 143 | Boyce et al. (2023), n = 175 | r = 0.22 | r = 0.28 [0.14, 0.41] | Held up | 2,933 |
| Affect heuristic | Judgment and decision-making | Finucane et al. (2000), n = 219 | Efendić et al. (2022), n = 776 | r = 0.74 | r = 0.87 [0.59, 0.96] | Held up | 2,870 |
| Elderly words make people walk slower | Priming and embodied cognition | Bargh et al. (1996), n = 30 | Doyen et al. (2012), n = 120 | r = 0.47 | r = -0.10 [-0.27, 0.08] | Failed | 2,736 |
| Dual n-back training raises fluid intelligence (brain training) | Reasoning, intelligence and mindset | Jaeggi et al. (2008), n = 69 | Redick et al. (2013), n = 73 | r = 0.27 | no single estimate (p = 0.68) | Failed | 2,183 |
| Most drivers think they are above average | Judgment and decision-making | Svenson (1981), n = 86 | Koppel et al. (2023), n = 1,203 | – | r = 0.81 [0.77, 0.84] | Held up | 1,969 |
| Holding a pen in your teeth makes cartoons funnier (facial feedback) | Priming and embodied cognition | Strack et al. (1988), n = not stated | Wagenmakers et al. (2016), n = 1,894 | 0.82 rating points | 0.03 [-0.11, 0.16] rating points | Failed | 1,894 |
| Spatial metaphors shape how we think about time | Priming and embodied cognition | Boroditsky (2000), n = 56 | Ebersole et al. (2016), n = 1,335 | r = 0.30 | r = 0.13 [0.07, 0.18] | Weaker | 1,841 |
| The “Google effect” on memory | Memory and learning | Sparrow et al. (2011), n = 69 | Camerer et al. (2018), n = 234 | r = 0.37 | r = 0.05 [-0.08, 0.18] | Failed | 1,744 |
| Ego depletion (willpower runs out) | Self-control and willpower | Muraven et al. (1998), n = not stated | Murtagh & Todd (2004), n = 76 | – | r = -0.09 [-0.31, 0.14] | Failed | 1,723 |
| Praising intelligence (not effort) makes children give up | Reasoning, intelligence and mindset | Mueller & Dweck (1998), n = 77 | Li & Bates (2019), n = 190 | r = 0.56 | r = 0.14 [0.00, 0.28] | Weaker | 1,703 |
| Reading literary fiction improves mind-reading | Reasoning, intelligence and mindset | Kidd & Castano (2013), n = 86 | Camerer et al. (2018), n = 714 | r = 0.27 | r = -0.03 [-0.10, 0.05] | Failed | 1,641 |
| Scarcity tires the mind for later tasks | Judgment and decision-making | Shah et al. (2012), n = 56 | Camerer et al. (2018), n = 619 | r = 0.27 | r = -0.01 [-0.09, 0.06] | Failed | 1,562 |
| Hostility priming | Priming and embodied cognition | Srull (2004), n = 96 | McCarthy et al. (2018), n = 5,610 | – | r = 0.03 [0.00, 0.06] | Weaker | 1,529 |
| Mental fatigue lowers physical endurance | Self-control and willpower | Marcora et al. (2009), n = 16 | Holgado et al. (2019), n = 30 | r = 0.17 | r = -0.06 [-0.41, 0.30] | Failed | 1,450 |
| Bias blind spot | Judgment and decision-making | Pronin et al. (2002), n = 24 | Chandrashekar et al. (2021), n = 621 | r = 0.87 | r = 0.79 [0.76, 0.81] | Held up | 1,285 |
| Babies learn abstract rules at 7 months | Babies and young children | Marcus et al. (1999), n = 16 | Spit et al. (2022), n = 96 | r = 0.80 | r = 0.04 [-0.16, 0.24] | Failed | 1,246 |
| Preschool self-control (the marshmallow test) predicts later achievement | Self-control and willpower | Shoda et al. (1990), n = 35 | Watts et al. (2018), n = 918 | r = 0.57 | r = 0.30 [0.24, 0.36] | Held up | 1,246 |
| Listening to Mozart boosts spatial reasoning (the Mozart effect) | Attention and thinking | Rauscher et al. (1993), n = 36 | Steele et al. (1999), n = 125 | – | r = 0.03 [-0.18, 0.24] | Failed | 1,235 |
| Low numeracy, bigger framing effect | Reasoning, intelligence and mindset | Peters et al. (2006), n = 100 | Zhu & Feldman (2023), n = 860 | r = 0.23 | r = 0.08 [0.01, 0.14] | Weaker | 1,178 |
| People rate good traits as more like themselves | Judgment and decision-making | Alicke (1985), n = 88 | Ziano et al. (2020), n = 1,573 | r = 0.88 | r = 0.73 [0.66, 0.81] | Held up | 1,046 |
| Retrieval practice beats concept mapping | Memory and learning | Karpicke & Blunt (2011), n = 40 | Camerer et al. (2018), n = 49 | r = 0.60 | r = 0.38 [0.12, 0.60] | Held up | 1,014 |
| Goals act as reference points | Judgment and decision-making | Heath et al. (1999), n = 45 | Au & Feldman (2024), n = 377 | r = 0.62 | r = 0.44 [0.40, 0.48] | Held up | 1,008 |
| Babies track what others believe | Babies and young children | Téglás & Endress (2010), n = 24 | Camerer et al. (2018), n = 95 | r = 0.45 | r = 0.59 [0.44, 0.70] | Held up | 923 |
| Deep thinkers adjust more from their own anchors | Judgment and decision-making | Epley & Gilovich (2006), n = 81 | Röseler et al. (2024), n = 303 | – | r = -0.05 [-0.16, 0.06] | Failed | 920 |
| Better than average at easy things, worse at hard ones | Judgment and decision-making | Kruger (1999), n = 37 | Korbmacher et al. (2022), n = 240 | r = 0.96 | r = 0.85 [0.37, 0.97] | Held up | 899 |
| Outcome bias: good results make decisions look good | Judgment and decision-making | Baron & Hershey (1988), n = 20 | Aiyer et al. (2023), n = 692 | r = 0.10 | r = 0.36 [0.30, 0.42] | Held up | 872 |
| Deep thinkers are swayed more by strong arguments | Reasoning, intelligence and mindset | Cacioppo et al. (1983), n = 114 | Ebersole et al. (2016), n = 2,365 | r = 0.41 | r = 0.01 [-0.03, 0.05] | Failed | 849 |
| Hard-to-read fonts improve reasoning | Reasoning, intelligence and mindset | Alter et al. (2007), n = 41 | Klein et al. (2018), n = 2,580 | r = 0.30 | r = -0.01 [-0.04, 0.01] | Failed | 844 |
| Emotion makes people ignore probability | Judgment and decision-making | Rottenstreich & Hsee (2001), n = 40 | Klein et al. (2018), n = 7,218 | r = 0.35 | r = -0.04 [-0.06, -0.01] | Failed | 825 |
| Describing a face makes it harder to recognise (verbal overshadowing) | Memory and learning | Schooler & Engstler-Schooler (1990), n = 88 | Alogna et al. (2014), n = 2,055 | 25.0 percentage points correct | 16.31 [12.14, 20.47] percentage points correct | Held up | 791 |
| A nearby smartphone drains brainpower | Attention and thinking | Ward et al. (2017), n = 275 | Ruiz Pardo & Minda (2022), n = 383 | r = 0.16 | r = 0.02 [-0.08, 0.12] | Failed | 783 |
| Anchoring in estimates | Judgment and decision-making | Jacowitz & Kahneman (1995), n = 104 | Klein et al. (2014), n = 5,627 | r = 0.42 | r = 0.74 [0.73, 0.75] | Held up | 753 |
| Feelings make people ignore quantity | Judgment and decision-making | Hsee & Rottenstreich (2004), n = 331 | Tam & Feldman (2024), n = 1,041 | r = 0.15 | r = 0.11 [0.05, 0.17] | Held up | 654 |
| Self-control tasks drain later self-control | Self-control and willpower | Schmeichel (2007), n = 141 | Carter & McCullough (2013), n = 235 | r = 0.21 | r = -0.15 [-0.27, -0.02] | Failed | 643 |
| Money illusion | Judgment and decision-making | Shafir et al. (1997), n = 358 | Ziano et al. (2021), n = 604 | r = 0.26 | r = 0.28 [0.21, 0.36] | Held up | 616 |
| Thinking about professors improves trivia scores | Priming and embodied cognition | Dijksterhuis & Knippenberg (1998), n = not stated | O’Donnell & Nelson (2018), n = 4,493 | 13.2 percentage points correct | 0.14 [-0.71, 1.0] percentage points correct | Failed | 584 |
| Disjunction effect | Judgment and decision-making | Tversky & Shafir (1992), n = 66 | Ziano et al. (2021), n = 445 | r = 0.22 | r = 0.30 [0.24, 0.37] | Held up | 581 |
| Weight in your hands sways how you judge a job candidate | Priming and embodied cognition | Ackerman et al. (2010), n = 54 | Camerer et al. (2018), n = 599 | r = 0.27 | r = 0.06 [-0.02, 0.14] | Failed | 575 |
| Choosing versus rejecting | Judgment and decision-making | Shafir (1993), n = 170 | Klein et al. (2018), n = 7,901 | r = 0.17 | r = -0.06 [-0.09, -0.04] | Failed | 572 |
| People find time alone with their thoughts unpleasant | Attention and thinking | Wilson et al. (2014), n = 30 | Camerer et al. (2018), n = 39 | r = 0.67 | r = 0.59 [0.34, 0.77] | Held up | 536 |
| Hindsight bias | Judgment and decision-making | Slovic & Fischhoff (1977), n = 184 | Chen et al. (2021), n = 608 | – | r = 0.20 [0.12, 0.27] | Held up | 488 |
| Unlikely events are thought of more abstractly | Reasoning, intelligence and mindset | Wakslak et al. (2006), n = 34 | Calderon et al. (2020), n = 120 | r = 0.52 | r = 0.02 [-0.16, 0.20] | Failed | 468 |
| Thinking about distance weakens felt bonds to family | Priming and embodied cognition | Williams & Bargh (2008), n = 84 | (2015). Estimating the reproducibility of psychological science. Science (6251), n = 125 | r = 0.30 | r = 0.05 [-0.12, 0.23] | Failed | 459 |
| The curse of knowledge | Reasoning, intelligence and mindset | Birch & Bloom (2007), n = 155 | Boyce et al. (2023), n = 225 | r = 0.23 | r = 0.10 [-0.03, 0.23] | Failed | 451 |
| Frequencies do not remove the conjunction fallacy | Judgment and decision-making | Mellers et al. (2001), n = 108 | Chandrashekar et al. (2021), n = 1,032 | r = 0.28 | r = 0.24 [0.15, 0.32] | Held up | 448 |
| The effort heuristic | Judgment and decision-making | Kruger et al. (2004), n = 138 | Ziano et al. (2023), n = 700 | r = 0.17 | r = 0.11 [0.04, 0.19] | Held up | 438 |
| Writing about worries boosts exam scores | Memory and learning | Ramirez & Beilock (2011), n = 20 | Camerer et al. (2018), n = 79 | r = 0.79 | r = -0.10 [-0.31, 0.13] | Failed | 433 |
| Scarcity changes how people judge value | Judgment and decision-making | Shah et al. (2015), n = 73 | Donnell et al. (2021), n = 207 | r = 0.26 | r = 0.09 [-0.05, 0.22] | Failed | 432 |
| Heavy clipboards make things seem important | Priming and embodied cognition | Jostmann et al. (2009), n = 51 | Ebersole et al. (2016), n = 2,285 | r = 0.28 | r = 0.01 [-0.03, 0.06] | Failed | 404 |
| Less is better | Judgment and decision-making | Hsee (1998), n = 83 | Klein et al. (2018), n = 7,646 | r = 0.33 | r = 0.36 [0.35, 0.38] | Held up | 404 |
Source: IQ Metrics Research replication scorecard [1]; citations from OpenAlex [2].
12 What this scorecard cannot tell you
Read every number with these in mind
- One replication is not the final word. A failed replication can miss a real effect, and some original authors dispute the replications.
- Famous is defined by citations (400 or more in OpenAlex). Findings that are famous in the media but rarely cited by scientists are not here.
- The set is not a random sample of science. It contains findings that someone thought worth replicating, which may favour doubtful ones.
- Converting every result to a correlation (r) is approximate, especially for tests with more than two groups or one-sample proportions.
- Confidence intervals for multi-lab replications are those printed in the report where available; elsewhere we computed them from r and n, which ignores differences between labs.
- Citations after a replication may criticise the finding. They show attention, not belief.
Curious about your own reasoning?
These findings are about groups of people in labs. Our IQ test measures reasoning across many kinds of questions and places your score against adults, with a percentile.
Take the IQ test13 How was the scorecard built?
- 01
Start from the replication database
The FORRT Replication Database (FReD, version of 1 February 2026, CC BY 4.0) [1] lists 2,164 original-and-replication pairs from psychology and nearby fields. We added 7 multi-site replications that are not in it, read from their reports.
- 02
Keep the famous ones
An original paper is famous if OpenAlex counts at least 400 citations (pulled 27 October 2026) [2].
- 03
Keep brain and intelligence topics
Memory, learning, attention, reasoning, intelligence and mindsets, self-control, judgment and decisions, priming and embodied cognition, and infant cognition. Left out: effects whose outcome is a moral, political, religious or prosocial judgment, attitudes to groups, personality correlations, and differences between groups of people (sex, race, class, income, age, culture).
- 04
Pick one replication per finding
The largest replication run by a team without the original authors and with at least as many participants as the original. If it tested several items, the one with the most participants, then the first listed.
- 05
Check every number in the papers
Each original and replication statistic was read in the paper or the replication report (the reports restate the original result when the original is paywalled), with the page noted in the CSV.
- 06
Put both on one scale
Every result was converted to a correlation r (from t, F with one degree of freedom, chi-square, Cohen’s d or h, odds ratios or eta-squared), with the original’s direction counted as positive. 95% CIs use Fisher’s z, unless the replication report printed its own interval.
- 07
Apply the rule
Failed: the replication’s 95% CI includes zero, or the effect went the other way. Weaker: same direction and the CI excludes zero, but the effect is below r = .10 (Cohen’s “small”) or less than half the original. Held up: everything else.
- 08
Sources
- FORRT, Open Science Framework (osf.io/9r62x). FORRT Replication Database (FReD) / FORRT Library of Replication Attempts (FLoRA): effect-level file FReD.xlsx (version of 1 February 2026). Wallrich, Röseler et al.. 2026. doi.org/10.17605/OSF.IO/9R62XCC BY 4.0
- OurResearch. OpenAlex: works and citations by year (pulled 27 October 2026). 2026. openalex.orgCC0
- Crossref. Crossref REST API: retraction, correction and expression-of-concern notices (checked 27 October 2026). 2026. api.crossref.orgMetadata openly available (CC0)
- . Lucca, K., Yuen, F., Wang, Y., et al. (2025). Infants' social evaluation of helpers and hinderers: A large-scale, multi-lab, coordinated replication study. Developmental Science, 28(1), e13581 (ManyBabies 4).. 2025. doi.org/10.1111/desc.13581Cited for its published result (facts); not reproduced
- . Coles, N. A., March, D. S., Marmolejo-Ramos, F., et al. (2022). A multi-lab test of the facial feedback hypothesis by the Many Smiles Collaboration. Nature Human Behaviour, 6, 1731-1742.. 2022. doi.org/10.1038/s41562-022-01458-9Cited for its published result (facts); not reproduced
- . Stroop, J. R. (1935). Studies of interference in serial verbal reactions. Journal of Experimental Psychology, 18(6), 643–662. https://doi.org/10.1037/h0054651. 1935. doi.org/10.1037/h0054651Cited for its published result (facts); not reproduced
- . Ebersole, C. R., Atherton, O. E., Belanger, A. L., Skulborstad, H. M., Allen, J. M., Banks, J. B., Baranski, E., Bernstein, M. J., Bonfiglio, D. B. V., Boucher, L., Brown, E. R., Budiman, N. I., Cairo, A. H., Capaldi, C. A., Chartier, C. R., Chung, J. M., Cicero, D. C., Coleman, J. A., Conway, J. G., … Nosek, B. A. (2016). Many Labs 3: Evaluating participant pool quality across the academic semester via replication. Journal of Experimental Social Psychology, 67, 68–82. https://doi.org/10.1016/j.jesp.2015.10.012. 2016. doi.org/10.1016/j.jesp.2015.10.012Cited for its published result (facts); not reproduced
- . Tversky, A., & Kahneman, D. (1981). The Framing of Decisions and the Psychology of Choice. Science, 211(4481), 453–458. https://doi.org/10.1126/science.7455683. 1981. doi.org/10.1126/science.7455683Cited for its published result (facts); not reproduced
- . Klein, R. A., Vianello, M., Hasselman, F., Adams, B. G., Adams, R. B., Alper, S., Aveyard, M., Axt, J. R., Babalola, M. T., Bahník, Š., Batra, R., Berkics, M., Bernstein, M. J., Berry, D. R., Bialobrzeska, O., Binan, E. D., Bocian, K., Brandt, M. J., Busching, R., … Nosek, B. A. (2018). Many Labs 2: Investigating Variation in Replicability Across Samples and Settings. Advances in Methods and Practices in Psychological Science, 1(4), 443–490. https://doi.org/10.1177/2515245918810225. 2018. doi.org/10.1177/2515245918810225Cited for its published result (facts); not reproduced
- . Tversky, A., & Kahneman, D. (1973). Availability: A heuristic for judging frequency and probability. Cognitive Psychology, 5(2), 207–232. https://doi.org/10.1016/0010-0285(73)90033-9. 1973. doi.org/10.1016/0010-0285(73)90033-9Cited for its published result (facts); not reproduced
- . Blackwell, L. S., Trzesniewski, K. H., & Dweck, C. S. (2007). Implicit Theories of Intelligence Predict Achievement Across an Adolescent Transition: A Longitudinal Study and an Intervention. Child Development, 78(1), 246–263. Portico. https://doi.org/10.1111/j.1467-8624.2007.00995.x. 2007. doi.org/10.1111/j.1467-8624.2007.00995.xCited for its published result (facts); not reproduced
- . Li, Y., & Bates, T. C. (2020). Testing the association of growth mindset and grades across a challenging transition: Is growth mindset associated with grades? Intelligence, 81, 101471. https://doi.org/10.1016/j.intell.2020.101471. 2020. doi.org/10.1016/j.intell.2020.101471Cited for its published result (facts); not reproduced
- . Ross, L., Greene, D., & House, P. (1977). The “false consensus effect”: An egocentric bias in social perception and attribution processes. Journal of Experimental Social Psychology, 13(3), 279–301. https://doi.org/10.1016/0022-1031(77)90049-x. 1977. doi.org/10.1016/0022-1031(77)90049-xCited for its published result (facts); not reproduced
- . Kahneman, D., & Miller, D. T. (1986). Norm theory: Comparing reality to its alternatives. Psychological Review, 93(2), 136–153. https://doi.org/10.1037/0033-295x.93.2.136. 1986. doi.org/10.1037/0033-295x.93.2.136Cited for its published result (facts); not reproduced
- . Kutscher, L., & Feldman, G. (2019). The impact of past behaviour normality on regret: replication and extension of three experiments of the exceptionality effect. Cognition and Emotion, 33(5), 901-914. https://doi.org/10.1080/02699931.2018.1504747. 2019. doi.org/10.1080/02699931.2018.1504747Cited for its published result (facts); not reproduced
- . Gick, M. L., & Holyoak, K. J. (1983). Schema induction and analogical transfer. Cognitive Psychology, 15(1), 1–38. https://doi.org/10.1016/0010-0285(83)90002-6. 1983. doi.org/10.1016/0010-0285(83)90002-6Cited for its published result (facts); not reproduced
- . Boyce, V., Mathur, M., & Frank, M. C. (2023). Eleven years of student replication projects provide evidence on the correlates of replicability in psychology. Royal Society Open Science, 10(11). https://doi.org/10.1098/rsos.231240. 2023. doi.org/10.1098/rsos.231240Cited for its published result (facts); not reproduced
- . Finucane, M. L., Alhakami, A., Slovic, P., & Johnson, S. M. (2000). The affect heuristic in judgments of risks and benefits. Journal of Behavioral Decision Making, 13(1), 1–17. https://doi.org/10.1002/(sici)1099-0771(200001/03)13:13.0.co;2-s. 2000. doi.org/10.1002/(sici)1099-0771(200001/03)13:1<1::aid-bdm333>3.0.co;2-sCited for its published result (facts); not reproduced
- . Efendić, E., Chandrashekar, S. P., Lee, C. S., Yeung, L. Y., Kim, M. J., Lee, C. Y., & Feldman, G. (2022). Risky therefore not beneficial: Replication and extension of Finucane et al.'s (2000) affect heuristic experiment. Social Psychological and Personality Science, 13(7), 1173-1184. https://doi.org/10.1177/19485506211056761. 2022. doi.org/10.1177/19485506211056761Cited for its published result (facts); not reproduced
- . Bargh, J. A., Chen, M., & Burrows, L. (1996). Automaticity of social behavior: Direct effects of trait construct and stereotype activation on action. Journal of Personality and Social Psychology, 71(2), 230–244. https://doi.org/10.1037/0022-3514.71.2.230. 1996. doi.org/10.1037/0022-3514.71.2.230Cited for its published result (facts); not reproduced
- . Doyen, S., Klein, O., Pichon, C.-L., & Cleeremans, A. (2012). Behavioral Priming: It’s All in the Mind, but Whose Mind? PLoS ONE, 7(1), e29081. https://doi.org/10.1371/journal.pone.0029081. 2012. doi.org/10.1371/journal.pone.0029081Cited for its published result (facts); not reproduced
- . Svenson, O. (1981). Are we all less risky and more skillful than our fellow drivers? Acta Psychologica, 47(2), 143–148. https://doi.org/10.1016/0001-6918(81)90005-6. 1981. doi.org/10.1016/0001-6918(81)90005-6Cited for its published result (facts); not reproduced
- . Koppel, L., Andersson, D., Tinghög, G., Västfjäll, D., & Feldman, G. (2023). We are all less risky and more skillful than our fellow drivers: Successful replication and extension of Svenson (1981). Meta-Psychology, 7. https://doi.org/10.15626/mp.2021.2932. 2023. doi.org/10.15626/mp.2021.2932Cited for its published result (facts); not reproduced
- . Boroditsky, L. (2000). Metaphoric structuring: understanding time through spatial metaphors. Cognition, 75(1), 1–28. https://doi.org/10.1016/s0010-0277(99)00073-6. 2000. doi.org/10.1016/s0010-0277(99)00073-6Cited for its published result (facts); not reproduced
- . Sparrow, B., Liu, J., & Wegner, D. M. (2011). Google Effects on Memory: Cognitive Consequences of Having Information at Our Fingertips. Science, 333(6043), 776–778. https://doi.org/10.1126/science.1207745. 2011. doi.org/10.1126/science.1207745Cited for its published result (facts); not reproduced
- . Camerer, C. F., Dreber, A., Holzmeister, F., Ho, T.-H., Huber, J., Johannesson, M., Kirchler, M., Nave, G., Nosek, B. A., Pfeiffer, T., Altmejd, A., Buttrick, N., Chan, T., Chen, Y., Forsell, E., Gampa, A., Heikensten, E., Hummer, L., Imai, T., … Wu, H. (2018). Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015. Nature Human Behaviour, 2(9), 637–644. https://doi.org/10.1038/s41562-018-0399-z. 2018. doi.org/10.1038/s41562-018-0399-zCited for its published result (facts); not reproduced
- . Muraven, M., Tice, D. M., & Baumeister, R. F. (1998). Self-control as a limited resource: Regulatory depletion patterns. Journal of Personality and Social Psychology, 74(3), 774–789. https://doi.org/10.1037/0022-3514.74.3.774. 1998. doi.org/10.1037/0022-3514.74.3.774Cited for its published result (facts); not reproduced
- . Kidd, D. C., & Castano, E. (2013). Reading Literary Fiction Improves Theory of Mind. Science, 342(6156), 377–380. https://doi.org/10.1126/science.1239918. 2013. doi.org/10.1126/science.1239918Cited for its published result (facts); not reproduced
- . Shah, A. K., Mullainathan, S., & Shafir, E. (2012). Some Consequences of Having Too Little. Science, 338(6107), 682–685. https://doi.org/10.1126/science.1222426. 2012. doi.org/10.1126/science.1222426Cited for its published result (facts); not reproduced
- . Pronin, E., Lin, D. Y., & Ross, L. (2002). The Bias Blind Spot: Perceptions of Bias in Self Versus Others. Personality and Social Psychology Bulletin, 28(3), 369–381. https://doi.org/10.1177/0146167202286008. 2002. doi.org/10.1177/0146167202286008Cited for its published result (facts); not reproduced
- . Chandrashekar, S. P., Yeung, S. K., Yau, K. C., Cheung, C. Y., Agarwal, T. K., Wong, C. Y. J., Pillai, T., Thirlwell, T. N., Leung, W. N., Tse, C., Li, Y. T., Cheng, B. L., Chan, H. Y. C., & Feldman, G. (2021). Agency and self-other asymmetries in perceived bias and shortcomings (subtitle shortened). Judgment and Decision Making, 16(6), 1392–1412. https://doi.org/10.1017/s1930297500008470. 2021. doi.org/10.1017/s1930297500008470Cited for its published result (facts); not reproduced
- . Marcus, G. F., Vijayan, S., Bandi Rao, S., & Vishton, P. M. (1999). Rule Learning by Seven-Month-Old Infants. Science, 283(5398), 77–80. https://doi.org/10.1126/science.283.5398.77. 1999. doi.org/10.1126/science.283.5398.77Cited for its published result (facts); not reproduced
- . Geambașu, A., Spit, S., van Renswoude, D., Blom, E., Fikkert, P. J. P. M., Hunnius, S., Junge, C. C. M. M., Verhagen, J., Visser, I., Wijnen, F., & Levelt, C. C. (2022). Robustness of the rule‐learning effect in 7‐month‐old infants: A close, multicenter replication of Marcus et al. (1999). Developmental Science, 26(1). Portico. https://doi.org/10.1111/desc.13244. 2022. doi.org/10.1111/desc.13244Cited for its published result (facts); not reproduced
- . Peters, E., Västfjäll, D., Slovic, P., Mertz, C. K., Mazzocco, K., & Dickert, S. (2006). Numeracy and Decision Making. Psychological Science, 17(5), 407–413. https://doi.org/10.1111/j.1467-9280.2006.01720.x. 2006. doi.org/10.1111/j.1467-9280.2006.01720.xCited for its published result (facts); not reproduced
- . Zhu, M., & Feldman, G. (2023). Revisiting the links between numeracy and decision making: Replication Registered Report of Peters et al. (2006) with an extension examining confidence. Collabra: Psychology, 9(1).. 2023. doi.org/10.1525/collabra.77608Cited for its published result (facts); not reproduced
- . Karpicke, J. D., & Blunt, J. R. (2011). Retrieval Practice Produces More Learning than Elaborative Studying with Concept Mapping. Science, 331(6018), 772–775. https://doi.org/10.1126/science.1199327. 2011. doi.org/10.1126/science.1199327Cited for its published result (facts); not reproduced
- . Heath, C., Larrick, R. P., & Wu, G. (1999). Goals as Reference Points. Cognitive Psychology, 38(1), 79–109. https://doi.org/10.1006/cogp.1998.0708. 1999. doi.org/10.1006/cogp.1998.0708Cited for its published result (facts); not reproduced
- . Au, I. N. Y., & Feldman, G. (2024). Revisiting "Goals as Reference Points": Replication and extensions of Heath et al. (1999). OSF. https://doi.org/10.17605/OSF.IO/WMQTB. 2024. doi.org/10.17605/osf.io/wmqtbCited for its published result (facts); not reproduced
- . Kovács, Á. M., Téglás, E., & Endress, A. D. (2010). The Social Sense: Susceptibility to Others’ Beliefs in Human Infants and Adults. Science, 330(6012), 1830–1834. https://doi.org/10.1126/science.1190792. 2010. doi.org/10.1126/science.1190792Cited for its published result (facts); not reproduced
- . Epley, N., & Gilovich, T. (2006). The Anchoring-and-Adjustment Heuristic. Psychological Science, 17(4), 311–318. https://doi.org/10.1111/j.1467-9280.2006.01704.x. 2006. doi.org/10.1111/j.1467-9280.2006.01704.xCited for its published result (facts); not reproduced
- . Röseler, L., Bögler, H. L., Koßmann, L., Krueger, S. M., Bickenbach, S. L. C., Bühler, R., della Guardia, J., Köppel, L.-M. A., Möhring, J., Ponader, S., Roßmaier, K., & Sing, J. (2024). Need for cognition, cognitive load, and forewarning do not moderate anchoring effects. A replication study of Epley & Gilovich (Journal of Behavioral Decision Making, 2005; Psychological Science, 2006). Journal of Comments and Replications in Economics, 3(2024-8). https://doi.org/10.18718/81781.38. 2024. doi.org/10.18718/81781.38Cited for its published result (facts); not reproduced
- . Alter, A. L., Oppenheimer, D. M., Epley, N., & Eyre, R. N. (2007). Overcoming intuition: Metacognitive difficulty activates analytic reasoning. Journal of Experimental Psychology: General, 136(4), 569–576. https://doi.org/10.1037/0096-3445.136.4.569. 2007. doi.org/10.1037/0096-3445.136.4.569Cited for its published result (facts); not reproduced
- . Rottenstreich, Y., & Hsee, C. K. (2001). Money, Kisses, and Electric Shocks: On the Affective Psychology of Risk. Psychological Science, 12(3), 185–190. https://doi.org/10.1111/1467-9280.00334. 2001. doi.org/10.1111/1467-9280.00334Cited for its published result (facts); not reproduced
- . Ward, A. F., Duke, K., Gneezy, A., & Bos, M. W. (2017). Brain Drain: The Mere Presence of One’s Own Smartphone Reduces Available Cognitive Capacity. Journal of the Association for Consumer Research, 2(2), 140–154. https://doi.org/10.1086/691462. 2017. doi.org/10.1086/691462Cited for its published result (facts); not reproduced
- . Ruiz Pardo, A. C., & Minda, J. P. (2022). Reexamining the “brain drain” effect: A replication of Ward et al. (2017). Acta Psychologica, 230, 103717. https://doi.org/10.1016/j.actpsy.2022.103717. 2022. doi.org/10.1016/j.actpsy.2022.103717Cited for its published result (facts); not reproduced
- . Jacowitz, K. E., & Kahneman, D. (1995). Measures of Anchoring in Estimation Tasks. Personality and Social Psychology Bulletin, 21(11), 1161–1166. https://doi.org/10.1177/01461672952111004. 1995. doi.org/10.1177/01461672952111004Cited for its published result (facts); not reproduced
- . Klein, R. A., Ratliff, K. A., Vianello, M., Adams, R. B., Bahník, Š., Bernstein, M. J., Bocian, K., Brandt, M. J., Brooks, B., Brumbaugh, C. C., Cemalcilar, Z., Chandler, J., Cheong, W., Davis, W. E., Devos, T., Eisner, M., Frankowska, N., Furrow, D., Galliani, E. M., … Nosek, B. A. (2014). Investigating Variation in Replicability. Social Psychology, 45(3), 142–152. https://doi.org/10.1027/1864-9335/a000178. 2014. doi.org/10.1027/1864-9335/a000178Cited for its published result (facts); not reproduced
- . Hsee, C. K., & Rottenstreich, Y. (2004). Music, Pandas, and Muggers: On the Affective Psychology of Value. Journal of Experimental Psychology: General, 133(1), 23–30. https://doi.org/10.1037/0096-3445.133.1.23. 2004. doi.org/10.1037/0096-3445.133.1.23Cited for its published result (facts); not reproduced
- . Tam, C. Y., & Feldman, G. (2024). Revisiting the affective psychology of value: Replication and extension of Hsee and Rottenstreich (2004) [Unpublished preprint]. OSF. https://osf.io/qbp4g/ (preregistration: https://doi.org/10.17605/OSF.IO/AHCPW). 2024. doi.org/10.17605/osf.io/ahcpwCited for its published result (facts); not reproduced
- . Schmeichel, B. J. (2007). Attention control, memory updating, and emotion regulation temporarily reduce the capacity for executive control. Journal of Experimental Psychology: General, 136(2), 241–255. https://doi.org/10.1037/0096-3445.136.2.241. 2007. doi.org/10.1037/0096-3445.136.2.241Cited for its published result (facts); not reproduced
- . Carter, E. C., & McCullough, M. E. (2013). After a pair of self-control-intensive tasks, sucrose swishing improves subsequent working memory performance. BMC Psychology, 1(1). https://doi.org/10.1186/2050-7283-1-22. 2013. doi.org/10.1186/2050-7283-1-22Cited for its published result (facts); not reproduced
- . Tversky, A., & Shafir, E. (1992). The Disjunction Effect in Choice under Uncertainty. Psychological Science, 3(5), 305–310. https://doi.org/10.1111/j.1467-9280.1992.tb00678.x. 1992. doi.org/10.1111/j.1467-9280.1992.tb00678.xCited for its published result (facts); not reproduced
- . Ziano, I., Kong, M. F., Kim, H. J., Liu, C. Y., Wong, S. C., Cheng, B. L., & Feldman, G. (2021). Replication: Revisiting Tversky and Shafir’s (1992) Disjunction Effect with an extension comparing between and within subject designs. Journal of Economic Psychology, 83, 102350. https://doi.org/10.1016/j.joep.2020.102350. 2021. doi.org/10.1016/j.joep.2020.102350Cited for its published result (facts); not reproduced
- . Ackerman, J. M., Nocera, C. C., & Bargh, J. A. (2010). Incidental Haptic Sensations Influence Social Judgments and Decisions. Science, 328(5986), 1712–1715. https://doi.org/10.1126/science.1189993. 2010. doi.org/10.1126/science.1189993Cited for its published result (facts); not reproduced
- . Shafir, E. (1993). Choosing versus rejecting: Why some options are both better and worse than others. Memory & Cognition, 21(4), 546–556. https://doi.org/10.3758/bf03197186. 1993. doi.org/10.3758/bf03197186Cited for its published result (facts); not reproduced
- . Wilson, T. D., Reinhard, D. A., Westgate, E. C., Gilbert, D. T., Ellerbeck, N., Hahn, C., Brown, C. L., & Shaked, A. (2014). Just think: The challenges of the disengaged mind. Science, 345(6192), 75–77. https://doi.org/10.1126/science.1250830. 2014. doi.org/10.1126/science.1250830Cited for its published result (facts); not reproduced
- . Wakslak, C. J., Trope, Y., Liberman, N., & Alony, R. (2006). Seeing the forest when entry is unlikely: Probability and the mental representation of events. Journal of Experimental Psychology: General, 135(4), 641–653. https://doi.org/10.1037/0096-3445.135.4.641. 2006. doi.org/10.1037/0096-3445.135.4.641Cited for its published result (facts); not reproduced
- . Calderon, S., Mac Giolla, E., Ask, K., & Granhag, P. A. (2020). Subjective likelihood and the construal level of future events: A replication study of Wakslak, Trope, Liberman, and Alony (2006). Journal of Personality and Social Psychology, 119(5), e27–e37. https://doi.org/10.1037/pspa0000214. 2020. doi.org/10.1037/pspa0000214Cited for its published result (facts); not reproduced
- . Birch, S. A. J., & Bloom, P. (2007). The Curse of Knowledge in Reasoning About False Beliefs. Psychological Science, 18(5), 382–386. https://doi.org/10.1111/j.1467-9280.2007.01909.x. 2007. doi.org/10.1111/j.1467-9280.2007.01909.xCited for its published result (facts); not reproduced
- . Kruger, J., Wirtz, D., Van Boven, L., & Altermatt, T. W. (2004). The effort heuristic. Journal of Experimental Social Psychology, 40(1), 91–98. https://doi.org/10.1016/s0022-1031(03)00065-9. 2004. doi.org/10.1016/s0022-1031(03)00065-9Cited for its published result (facts); not reproduced
- . Ziano, I., Yeung, S. K., Lee, C. S., Shi, J., & Feldman, G. (2023). "The effort heuristic" revisited: Mixed results for replications of Kruger et al. (2004)'s Experiments 1 and 2. Collabra: Psychology, 9(1), 87489. https://doi.org/10.1525/collabra.87489. 2023. doi.org/10.1525/collabra.87489Cited for its published result (facts); not reproduced
- . Ramirez, G., & Beilock, S. L. (2011). Writing About Testing Worries Boosts Exam Performance in the Classroom. Science, 331(6014), 211–213. https://doi.org/10.1126/science.1199427. 2011. doi.org/10.1126/science.1199427Cited for its published result (facts); not reproduced
- . Jostmann, N. B., Lakens, D., & Schubert, T. W. (2009). Weight as an Embodiment of Importance. Psychological Science, 20(9), 1169–1174. https://doi.org/10.1111/j.1467-9280.2009.02426.x. 2009. doi.org/10.1111/j.1467-9280.2009.02426.xCited for its published result (facts); not reproduced
- . Hsee, C. K. (1998). Less is better: when low-value options are valued more highly than high-value options. Journal of Behavioral Decision Making, 11(2), 107–121. https://doi.org/10.1002/(sici)1099-0771(199806)11:23.0.co;2-y. 1998. doi.org/10.1002/(sici)1099-0771(199806)11:2<107::aid-bdm292>3.0.co;2-yCited for its published result (facts); not reproduced
- . Marcora, S. M., Staiano, W., & Manning, V. (2009). Mental fatigue impairs physical performance in humans. Journal of Applied Physiology, 106(3), 857–864. https://doi.org/10.1152/japplphysiol.91324.2008. 2009. doi.org/10.1152/japplphysiol.91324.2008Cited for its published result (facts); not reproduced
- . Holgado, D., Troya, E., Perales, J. C., Vadillo, M. A., & Sanabria, D. (2019). Does mental fatigue impair physical performance? A replication study. https://doi.org/10.31236/osf.io/tzn6d. 2019. doi.org/10.31236/osf.io/tzn6dCited for its published result (facts); not reproduced
- . Shafir, E., Diamond, P., & Tversky, A. (1997). Money Illusion. The Quarterly Journal of Economics, 112(2), 341–374. https://doi.org/10.1162/003355397555208. 1997. doi.org/10.1162/003355397555208Cited for its published result (facts); not reproduced
- . Ziano, I., Li, J., Tsun, S. M., Lei, H. C., Kamath, A. A., Cheng, B. L., & Feldman, G. (2021). Revisiting “money illusion”: Replication and extension of Shafir, Diamond, and Tversky (1997). Journal of Economic Psychology, 83, 102349. https://doi.org/10.1016/j.joep.2020.102349. 2021. doi.org/10.1016/j.joep.2020.102349Cited for its published result (facts); not reproduced
- . Mellers, B., Hertwig, R., & Kahneman, D. (2001). Do Frequency Representations Eliminate Conjunction Effects? An Exercise in Adversarial Collaboration. Psychological Science, 12(4), 269–275. https://doi.org/10.1111/1467-9280.00350. 2001. doi.org/10.1111/1467-9280.00350Cited for its published result (facts); not reproduced
- . Chandrashekar, S., Cheng, Y. H., Fong, C. L., Leung, Y. C., Wong, Y. T., Cheng, B. L., & Feldman, G. (2021). Frequency estimation and semantic ambiguity do not eliminate conjunction bias, when it occurs: Replication and extension of Mellers, Hertwig, and Kahneman (2001). Meta-Psychology, 5. https://doi.org/10.15626/mp.2020.2474. 2021. doi.org/10.15626/mp.2020.2474Cited for its published result (facts); not reproduced
- . Shah, A. K., Shafir, E., & Mullainathan, S. (2015). Scarcity Frames Value. Psychological Science, 26(4), 402–412. https://doi.org/10.1177/0956797614563958. 2015. doi.org/10.1177/0956797614563958Cited for its published result (facts); not reproduced
- . O’Donnell, M., Dev, A. S., Antonoplis, S., Baum, S. M., Benedetti, A. H., Brown, N. D., Carrillo, B., Choi, A. L., Connor, P., Donnelly, K., Ellwood-Lowe, M. E., Foushee, R., Jansen, R., Jarvis, S. N., Lundell-Creagh, R., Ocampo, J. M., Okafor, G. N., Azad, Z. R., Rosenblum, M., … Nelson, L. D. (2021). Empirical audit and review and an assessment of evidentiary value in research on the psychological consequences of scarcity. Proceedings of the National Academy of Sciences, 118(44). https://doi.org/10.1073/pnas.2103313118. 2021. doi.org/10.1073/pnas.2103313118Cited for its published result (facts); not reproduced
- . Williams, L. E., & Bargh, J. A. (2008). Keeping One’s Distance. Psychological Science, 19(3), 302–308. https://doi.org/10.1111/j.1467-9280.2008.02084.x. 2008. doi.org/10.1111/j.1467-9280.2008.02084.xCited for its published result (facts); not reproduced
- . (2015). Estimating the reproducibility of psychological science. Science, 349(6251). https://doi.org/10.1126/science.aac4716. 2015. doi.org/10.1126/science.aac4716Cited for its published result (facts); not reproduced
- . Cacioppo, J. T., Petty, R. E., & Morris, K. J. (1983). Effects of need for cognition on message evaluation, recall, and persuasion. Journal of Personality and Social Psychology, 45(4), 805–818. https://doi.org/10.1037/0022-3514.45.4.805. 1983. doi.org/10.1037/0022-3514.45.4.805Cited for its published result (facts); not reproduced
- . Alicke, M. D. (1985). Global self-evaluation as determined by the desirability and controllability of trait adjectives. Journal of Personality and Social Psychology, 49(6), 1621–1630. https://doi.org/10.1037/0022-3514.49.6.1621. 1985. doi.org/10.1037/0022-3514.49.6.1621Cited for its published result (facts); not reproduced
- . Ziano, I., Mok, P. Y. (Cora), & Feldman, G. (2020). Replication and Extension of Alicke (1985) Better-Than-Average Effect for Desirable and Controllable Traits. Social Psychological and Personality Science, 12(6), 1005–1017. https://doi.org/10.1177/1948550620948973. 2020. doi.org/10.1177/1948550620948973Cited for its published result (facts); not reproduced
- . Baron, J., & Hershey, J. C. (1988). Outcome bias in decision evaluation. Journal of Personality and Social Psychology, 54(4), 569–579. https://doi.org/10.1037/0022-3514.54.4.569. 1988. doi.org/10.1037/0022-3514.54.4.569Cited for its published result (facts); not reproduced
- . Aiyer, S., Kam, H. C., Ng, K. Y., Young, N. A., Shi, J., & Feldman, G. (2023). Outcomes affect evaluations of decision quality: Replication and extensions of Baron and Hershey's (1988) outcome bias Experiment 1. International Review of Social Psychology, 36(1).. 2023. doi.org/10.5334/irsp.751Cited for its published result (facts); not reproduced
- . Kruger, J. (1999). Lake Wobegon be gone! The “below-average effect” and the egocentric nature of comparative ability judgments. Journal of Personality and Social Psychology, 77(2), 221–232. https://doi.org/10.1037/0022-3514.77.2.221. 1999. doi.org/10.1037/0022-3514.77.2.221Cited for its published result (facts); not reproduced
- . Korbmacher, M., Kwan, C. I., & Feldman, G. (2022). Both better and worse than others depending on difficulty: Replication and extensions of Kruger's (1999) above and below average effects. Judgment and Decision Making, 17(2), 449-486. https://doi.org/10.1017/S1930297500009189. 2022. doi.org/10.1017/S1930297500009189Cited for its published result (facts); not reproduced
- . Slovic, P., & Fischhoff, B. (1977). On the psychology of experimental surprises. Journal of Experimental Psychology: Human Perception and Performance, 3(4), 544–551. https://doi.org/10.1037/0096-1523.3.4.544. 1977. doi.org/10.1037/0096-1523.3.4.544Cited for its published result (facts); not reproduced
- . Chen, J., Kwan, L. C., Ma, L. Y., Choi, H. Y., Lo, Y. C., Au, S. Y., Tsang, C. H., Cheng, B. L., & Feldman, G. (2021). Retrospective and prospective hindsight bias: Replications and extensions of Fischhoff (1975) and Slovic and Fischhoff (1977). Journal of Experimental Social Psychology, 96, 104154. https://doi.org/10.1016/j.jesp.2021.104154. 2021. doi.org/10.1016/j.jesp.2021.104154Cited for its published result (facts); not reproduced
- . Srull, T. K., & Wyer, Jr., R. S. (2004). The Role of Category Accessibility in the Interpretation of Information About Persons: Some Determinants and Implications. Social Cognition, 154–166. https://doi.org/10.4324/9780203496398-9. 2004. doi.org/10.1037/0022-3514.37.10.1660Cited for its published result (facts); not reproduced
- . McCarthy, R. J., Skowronski, J. J., Verschuere, B., Meijer, E. H., Jim, A., Hoogesteyn, K., Orthey, R., Acar, O. A., Aczel, B., Bakos, B. E., Barbosa, F., Baskin, E., Bègue, L., Ben-Shakhar, G., Birt, A. R., Blatz, L., Charman, S. D., Claesen, A., Clay, S. L., … Yıldız, E. (2018). Registered Replication Report on Srull and Wyer (1979). Advances in Methods and Practices in Psychological Science, 1(3), 321–336. https://doi.org/10.1177/2515245918777487. 2018. doi.org/10.1177/2515245918777487Cited for its published result (facts); not reproduced
- . Strack, F., Martin, L. L., & Stepper, S. (1988). Inhibiting and facilitating conditions of the human smile: A nonobtrusive test of the facial feedback hypothesis. Journal of Personality and Social Psychology, 54(5), 768-777.. 1988. doi.org/10.1037/0022-3514.54.5.768Cited for its published result (facts); not reproduced
- . Wagenmakers, E.-J., Beek, T., Dijkhoff, L., Gronau, Q. F., Acosta, A., Adams, R. B., Jr., … Zwaan, R. A. (2016). Registered Replication Report: Strack, Martin, & Stepper (1988). Perspectives on Psychological Science, 11(6), 917-928.. 2016. doi.org/10.1177/1745691616674458Cited for its published result (facts); not reproduced
- . Dijksterhuis, A., & van Knippenberg, A. (1998). The relation between perception and behavior, or how to win a game of Trivial Pursuit. Journal of Personality and Social Psychology, 74(4), 865-877.. 1998. doi.org/10.1037/0022-3514.74.4.865Cited for its published result (facts); not reproduced
- . O'Donnell, M., Nelson, L. D., et al. (2018). Registered Replication Report: Dijksterhuis and van Knippenberg (1998). Perspectives on Psychological Science, 13(2), 268-294.. 2018. doi.org/10.1177/1745691618755704Cited for its published result (facts); not reproduced
- . Schooler, J. W., & Engstler-Schooler, T. Y. (1990). Verbal overshadowing of visual memories: Some things are better left unsaid. Cognitive Psychology, 22(1), 36-71.. 1990. doi.org/10.1016/0010-0285(90)90003-mCited for its published result (facts); not reproduced
- . Alogna, V. K., Attaya, M. K., Aucoin, P., Bahnik, S., Birch, S., Birt, A. R., … Zwaan, R. A. (2014). Registered Replication Report: Schooler and Engstler-Schooler (1990). Perspectives on Psychological Science, 9(5), 556-578.. 2014. doi.org/10.1177/1745691614545653Cited for its published result (facts); not reproduced
- . Shoda, Y., Mischel, W., & Peake, P. K. (1990). Predicting adolescent cognitive and self-regulatory competencies from preschool delay of gratification: Identifying diagnostic conditions. Developmental Psychology, 26(6), 978-986.. 1990. doi.org/10.1037/0012-1649.26.6.978Cited for its published result (facts); not reproduced
- . Watts, T. W., Duncan, G. J., & Quan, H. (2018). Revisiting the marshmallow test: A conceptual replication investigating links between early delay of gratification and later outcomes. Psychological Science, 29(7), 1159-1177.. 2018. doi.org/10.1177/0956797618761661Cited for its published result (facts); not reproduced
- . Mueller, C. M., & Dweck, C. S. (1998). Praise for intelligence can undermine children's motivation and performance. Journal of Personality and Social Psychology, 75(1), 33-52.. 1998. doi.org/10.1037/0022-3514.75.1.33Cited for its published result (facts); not reproduced
- . Li, Y., & Bates, T. C. (2019). You can't change your basic ability, but you work at things, and that's how we get hard things done: Testing the role of growth mindset on response to setbacks, educational attainment, and cognitive ability. Journal of Experimental Psychology: General, 148(9), 1640-1655.. 2019. doi.org/10.1037/xge0000669Cited for its published result (facts); not reproduced
- . Jaeggi, S. M., Buschkuehl, M., Jonides, J., & Perrig, W. J. (2008). Improving fluid intelligence with training on working memory. Proceedings of the National Academy of Sciences, 105(19), 6829-6833.. 2008. doi.org/10.1073/pnas.0801268105Cited for its published result (facts); not reproduced
- . Redick, T. S., Shipstead, Z., Harrison, T. L., Hicks, K. L., Fried, D. E., Hambrick, D. Z., Kane, M. J., & Engle, R. W. (2013). No evidence of intelligence improvement after working memory training: A randomized, placebo-controlled study. Journal of Experimental Psychology: General, 142(2), 359-379.. 2013. doi.org/10.1037/a0029082Cited for its published result (facts); not reproduced
- . Rauscher, F. H., Shaw, G. L., & Ky, K. N. (1993). Music and spatial task performance. Nature, 365, 611.. 1993. doi.org/10.1038/365611a0Cited for its published result (facts); not reproduced
- . Steele, K. M., Bass, K. E., & Crook, M. D. (1999). The mystery of the Mozart effect: Failure to replicate. Psychological Science, 10(4), 366-369.. 1999. doi.org/10.1111/1467-9280.00169Cited for its published result (facts); not reproduced
Version history
- 2026.1027 October 2026First release.
Leon Falkner, Data & Research Analyst. Leon owns the arithmetic on IQ Metrics: the percentile tables, the score converter and the figures in everyone else's drafts. On the research desk Leon pulls the data, builds each index and writes the method. More about Leon.
Questions
What is the replication crisis?
The finding, since about 2011, that many published results in psychology and other fields do not appear again when other teams repeat the studies with larger samples. Large projects such as Many Labs and the Registered Replication Reports have tested dozens of famous effects.
How many famous psychology findings fail to replicate?
In our scorecard of 56 famous brain and intelligence findings, 26 (46%) failed their largest independent replication, 6 came back weaker and 24 (43%) held up. Large projects such as Many Labs and the Registered Replication Reports have retested dozens of famous effects.
Which famous brain findings held up?
Mostly classic judgment and decision-making effects: anchoring, the false consensus effect, outcome bias and the "less is better" effect came back at or above their original size. From memory research, retrieval practice and verbal overshadowing held up, and so did the Stroop effect, one of the oldest findings in psychology.
Did the marshmallow test replicate?
Partly. A 2018 study of about 900 children found that waiting longer for a second treat predicted later school achievement, but the link was about half as strong as in the famous 1990 study and shrank further after family background was taken into account.
Do scientists stop citing a finding after it fails to replicate?
Not in our data. In the 3 years after a failed replication, citations of the original changed by a median +18%, against +8% for findings that held up. Some of those citations criticise the finding, but it stays in the literature.
Use the data
Share it, cite it, download it
You may reuse everything on this page under CC BY 4.0: in articles, reports, lessons and posts. The one condition is credit to IQ Metrics with a link to this page.
Cite this study
Version 2026.10. Citing the version lets readers find the exact numbers.
Archived copy with a DOI: 10.5281/zenodo.23688461 (report PDF and data on Zenodo).
Falkner, L. (2026). Which famous brain and intelligence findings held up? (Version 2026.10) [Data set]. IQ Metrics Research. https://iqmetrics.org/research/replication-scorecard/ (DOI: 10.5281/zenodo.23688461)
Falkner, Leon. "Which famous brain and intelligence findings held up?" IQ Metrics Research, 27 Oct. 2026, iqmetrics.org/research/replication-scorecard/. DOI: 10.5281/zenodo.23688461.
Falkner, Leon. "Which famous brain and intelligence findings held up?" IQ Metrics Research. October 27, 2026. https://iqmetrics.org/research/replication-scorecard/. DOI: 10.5281/zenodo.23688461.
@misc{iqmetrics2026replicationscorecard, author = {Falkner, Leon}, title = {Which famous brain and intelligence findings held up?}, institution = {IQ Metrics}, year = {2026}, version = {2026.10}, doi = {10.5281/zenodo.23688461}, url = {https://iqmetrics.org/research/replication-scorecard/} }
Source: Leon Falkner, "Which famous brain and intelligence findings held up?" IQ Metrics Research, version 2026.10, https://iqmetrics.org/research/replication-scorecard/ (DOI 10.5281/zenodo.23688461)
Get the data
One row per famous finding (original cited 400+ times): the original and replication references, DOIs, years, sample sizes, the statistics as printed, both effects as a correlation r with 95% CIs, the outcome by our rule, the replication team’s verdict and citations before and after.
| Column | What it holds |
|---|---|
slug, finding, topic, outcome | Our id, a plain name for the finding, its topic, and held / weaker / failed by the stated rule |
original_reference, original_doi, original_year, original_n, original_statistic | The original study and its result as printed |
original_r, original_ci_low, original_ci_high | The original result as a correlation with its 95% CI (blank if not on a comparable scale) |
replication_reference, replication_doi, replication_year, replication_n, replication_statistic | The replication and its result as printed |
replication_r, replication_ci_low, replication_ci_high, replication_ci_source | The replication result as r, its 95% CI, and whether the CI was printed or computed |
size_ratio | Replication r divided by original r |
replication_team_verdict | The replication team's own verdict as coded in FReD (successful, mixed, failed) |
pair_source | FReD effect id, or the multi-site report it was read from |
citations_total, cites_before, cites_after, cite_change_pct | OpenAlex citations: all-time, average per year in the 3 years before and after the replication, and the change |
crossref_notices, openalex_id, checked, notes | Retraction or correction notices, the OpenAlex work id, where each number was checked, and notes |
Press kit
For journalists, researchers and teachers
Everything on this page may be quoted, charted and republished with credit. No embargo, no permission needed, no sign-up. If you need a number that is not here, ask.
Summary you can paste
Plain news style. Edit freely; keep the credit.
An IQ Metrics Research scorecard matched 56 famous findings in brain and cognitive science (each cited 400+ times) with their largest independent replications, published 1999-2024. 24 (43%) held up, 6 came back weaker and 26 failed. Replication effects were typically 62% smaller than the originals, and failed findings kept being cited: a median +18% in the 3 years after the replication. Source: IQ Metrics Research, https://iqmetrics.org/research/replication-scorecard/
Angles for a story
- How many famous findings held up (Figure 1).
- Every finding, original vs replication (Figure 2).
- Failed but still cited (citations after failure).
- Small studies, big promises (the bubble chart).
Quotes you can use
Attributable as written, with the name and title. For a different angle or a quote on a specific figure, ask through the contact page.
A famous finding is not the same as a solid one. Only about 4 in 10 of the findings we scored held up when an independent team ran them again.
Leon Falkner, Data & Research Analyst, IQ Metrics ResearchThe striking result is what happens next: a failed replication barely dents a finding’s citations. Science corrects itself, but slowly.
Leon Falkner, Data & Research Analyst, IQ Metrics ResearchThe classic decision-making effects, like anchoring, are as strong as ever. The fragile ones are the subtle nudges: priming, embodiment and willpower effects.
Leon Falkner, Data & Research Analyst, IQ Metrics Research
Files
- CSV of every finding, with both statistics, DOIs and citations (CC BY 4.0)
- Every figure as a print-ready PNG: use the Image button under each figure
- Share images for social media in three sizes, with the credit line built in
- Citations in APA, MLA, Chicago, BibTeX and plain text
Terms, contact and corrections
Credit: “IQ Metrics Research, Replication Scorecard” with a link to this page. On air or in print, “IQ Metrics” and the page address are enough.
Data credit: replication pairs from the FORRT Replication Database (CC BY 4.0) and the replication reports listed under Sources; citations from OpenAlex. The selection, checks, outcome rule and charts are ours.
Please say “failed to replicate” rather than “debunked” or “fake”: a failed replication is evidence against a finding, not proof of misconduct.
Contact: the research team, through the contact page. Corrections are made on this page with a dated note in the version history.
Keep reading
