Top performers are not the most impressive when extreme performance indicates unreliability - PubMed (original) (raw)
Top performers are not the most impressive when extreme performance indicates unreliability
Jerker Denrell et al. Proc Natl Acad Sci U S A. 2012.
Abstract
The relationship between performance and ability is a central concern in the social sciences: Are the most successful much more able than others, and are failures unskilled? Prior research has shown that noise and self-reinforcing dynamics make performance unpredictable and lead to a weak association between ability and performance. Here we show that the same mechanisms that generate unpredictability imply that extreme performances can be relatively uninformative about ability. As a result, the highest performers may not have the highest expected ability and should not be imitated or praised. We show that whether higher performance indicates higher ability depends on whether extreme performance could be achieved by skill or requires luck.
Conflict of interest statement
The authors declare no conflict of interest.
Figures
Fig. 1.
The average value of ci for players who obtained different numbers of successes in 50 rounds. Based on 5 million simulations.
Fig. 2.
Illustration of the association between skill and the number of successes obtained in 50 rounds for strong and weak dependence. (A) wi = 0.9. (B) wi = 0.1. Each figure is based on 1,000 simulations.
Fig. 3.
How expected degree of skill, E[ui|_Pi_], varies with performance when there is heterogeneity in both the skill and noise distribution but the noise distribution is more fat-tailed (based on numerical integration, for the case when n = 1 and s = 5).
Fig. 4.
Illustration of the association between skill and performance when the SD of the noise term is high and low. (A) The SD of the noise term is 3. (B) The SD of the noise term is 0.5. Each figure is based on 1,000 simulations.
Similar articles
- Skilled or unskilled, but still unaware of it: how perceptions of difficulty drive miscalibration in relative comparisons.
Burson KA, Larrick RP, Klayman J. Burson KA, et al. J Pers Soc Psychol. 2006 Jan;90(1):60-77. doi: 10.1037/0022-3514.90.1.60. J Pers Soc Psychol. 2006. PMID: 16448310 - The contribution of judgment scale to the unskilled-and-unaware phenomenon: how evaluating others can exaggerate over- (and under-) confidence.
Hartwig MK, Dunlosky J. Hartwig MK, et al. Mem Cognit. 2014 Jan;42(1):164-73. doi: 10.3758/s13421-013-0351-4. Mem Cognit. 2014. PMID: 23846414 - Skill and self-knowledge: empirical refutation of the dual-burden account of the Dunning-Kruger effect.
McIntosh RD, Moore AB, Liu Y, Della Sala S. McIntosh RD, et al. R Soc Open Sci. 2022 Dec 7;9(12):191727. doi: 10.1098/rsos.191727. eCollection 2022 Dec. R Soc Open Sci. 2022. PMID: 36483762 Free PMC article. - Neonatal Monitoring: Prediction of Autonomic Regulation at 1 Month from Newborn Assessments.
Myers MM, Burtchen N, Retamar MO, Lucchini M, Fifer WP. Myers MM, et al. In: Duncan JR, Byard RW, editors. SIDS Sudden Infant and Early Childhood Death: The Past, the Present and the Future. Adelaide (AU): University of Adelaide Press; 2018 May. Chapter 21. In: Duncan JR, Byard RW, editors. SIDS Sudden Infant and Early Childhood Death: The Past, the Present and the Future. Adelaide (AU): University of Adelaide Press; 2018 May. Chapter 21. PMID: 30035942 Free Books & Documents. Review. - Anxiety and sport performance.
Raglin JS. Raglin JS. Exerc Sport Sci Rev. 1992;20:243-74. Exerc Sport Sci Rev. 1992. PMID: 1623887 Review.
Cited by
- Imitation versus serendipity in ranking dynamics.
De Domenico F, Caccioli F, Livan G, Montagna G, Nicrosini O. De Domenico F, et al. R Soc Open Sci. 2024 Jul 24;11(7):240177. doi: 10.1098/rsos.240177. eCollection 2024 Jul. R Soc Open Sci. 2024. PMID: 39050725 Free PMC article. - A normative theory of luck.
Liu C, Tsay CJ. Liu C, et al. Front Psychol. 2023 Nov 10;14:1157527. doi: 10.3389/fpsyg.2023.1157527. eCollection 2023. Front Psychol. 2023. PMID: 38022948 Free PMC article. - The relative importance of ability, luck and motivation in team sports: a Bayesian model of performance in the English Rugby Premiership.
Fioravanti F, Delbianco F, Tohmé F. Fioravanti F, et al. Stat Methods Appt. 2022 Dec 8:1-17. doi: 10.1007/s10260-022-00677-8. Online ahead of print. Stat Methods Appt. 2022. PMID: 36532203 Free PMC article. - On the role of chance in fencing tournaments: An agent-based approach.
Zappalà C, Pluchino A, Rapisarda A, Biondo AE, Sobkowicz P. Zappalà C, et al. PLoS One. 2022 May 5;17(5):e0267541. doi: 10.1371/journal.pone.0267541. eCollection 2022. PLoS One. 2022. PMID: 35511768 Free PMC article. - Don't follow the leader: how ranking performance reduces meritocracy.
Livan G. Livan G. R Soc Open Sci. 2019 Nov 6;6(11):191255. doi: 10.1098/rsos.191255. eCollection 2019 Nov. R Soc Open Sci. 2019. PMID: 31827860 Free PMC article.
References
- Baron J, Hershey JC. Outcome bias in decision evaluation. J Pers Soc Psychol. 1988;54(4):569–579. - PubMed
- Gilbert DT, Malone PS. The correspondence bias. Psychol Bull. 1995;117(1):21–38. - PubMed
- Richerson PJ, Boyd R. Not by Genes Alone: How Culture Transformed Human Evolution. Chicago: Univ of Chicago Press; 2005.
- Rogers AR. Does biology constrain culture? Am Anthropol. 1988;90(4):819–831.
- Thorngate W, Dawes R, Foddy M. Judging Merit. New York, NY: Psychology Press; 2008.
LinkOut - more resources
Full Text Sources
Research Materials