Nonsignificance plus high power does not imply support for the null over the alternative - PubMed (original) (raw)
Nonsignificance plus high power does not imply support for the null over the alternative
Sander Greenland. Ann Epidemiol. 2012 May.
Abstract
This article summarizes arguments against the use of power to analyze data, and illustrates a key pitfall: Lack of statistical significance (e.g., p > .05) combined with high power (e.g., 90%) can occur even if the data support the alternative more than the null. This problem arises via selective choice of parameters at which power is calculated, but can also arise if one computes power at a prespecified alternative. As noted by earlier authors, power computed using sample estimates ("observed power") replaces this problem with even more counterintuitive behavior, because observed power effectively double counts the data and increases as the P value declines. Use of power to analyze and interpret data thus needs more extensive discouragement.
Copyright © 2012 Elsevier Inc. All rights reserved.
Similar articles
- Null misinterpretation in statistical testing and its impact on health risk assessment.
Greenland S. Greenland S. Prev Med. 2011 Oct;53(4-5):225-8. doi: 10.1016/j.ypmed.2011.08.010. Epub 2011 Aug 17. Prev Med. 2011. PMID: 21871481 - Impact of criticism of null-hypothesis significance testing on statistical reporting practices in conservation biology.
Fidler F, Burgman MA, Cumming G, Buttrose R, Thomason N. Fidler F, et al. Conserv Biol. 2006 Oct;20(5):1539-44. doi: 10.1111/j.1523-1739.2006.00525.x. Conserv Biol. 2006. PMID: 17002771 - The Bayesian interpretation of a P-value depends only weakly on statistical power in realistic situations.
Hooper R. Hooper R. J Clin Epidemiol. 2009 Dec;62(12):1242-7. doi: 10.1016/j.jclinepi.2009.02.004. Epub 2009 Apr 23. J Clin Epidemiol. 2009. PMID: 19398295 - Seven ways to increase power without increasing N.
Hansen WB, Collins LM. Hansen WB, et al. NIDA Res Monogr. 1994;142:184-95. NIDA Res Monogr. 1994. PMID: 9243537 Review. - Effect of formal statistical significance on the credibility of observational associations.
Ioannidis JP. Ioannidis JP. Am J Epidemiol. 2008 Aug 15;168(4):374-83; discussion 384-90. doi: 10.1093/aje/kwn156. Epub 2008 Jul 8. Am J Epidemiol. 2008. PMID: 18611956 Review.
Cited by
- Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations.
Greenland S, Senn SJ, Rothman KJ, Carlin JB, Poole C, Goodman SN, Altman DG. Greenland S, et al. Eur J Epidemiol. 2016 Apr;31(4):337-50. doi: 10.1007/s10654-016-0149-3. Epub 2016 May 21. Eur J Epidemiol. 2016. PMID: 27209009 Free PMC article. - A CHecklist for statistical Assessment of Medical Papers (the CHAMP statement): explanation and elaboration.
Mansournia MA, Collins GS, Nielsen RO, Nazemipour M, Jewell NP, Altman DG, Campbell MJ. Mansournia MA, et al. Br J Sports Med. 2021 Sep;55(18):1009-1017. doi: 10.1136/bjsports-2020-103652. Epub 2021 Jan 29. Br J Sports Med. 2021. PMID: 33514558 Free PMC article. - Increasing value and reducing waste in research design, conduct, and analysis.
Ioannidis JP, Greenland S, Hlatky MA, Khoury MJ, Macleod MR, Moher D, Schulz KF, Tibshirani R. Ioannidis JP, et al. Lancet. 2014 Jan 11;383(9912):166-75. doi: 10.1016/S0140-6736(13)62227-8. Epub 2014 Jan 8. Lancet. 2014. PMID: 24411645 Free PMC article. - The earth is flat (p > 0.05): significance thresholds and the crisis of unreplicable research.
Amrhein V, Korner-Nievergelt F, Roth T. Amrhein V, et al. PeerJ. 2017 Jul 7;5:e3544. doi: 10.7717/peerj.3544. eCollection 2017. PeerJ. 2017. PMID: 28698825 Free PMC article. - COVID-19 stressors and symptoms of depression and anxiety among Black cisgender sexual minority men and Black transgender women during the initial peak of the COVID-19 pandemic.
Timmins L, Schneider JA, Chen YT, Pagkas-Bather J, Kim B, Moody RL, Al-Ajlouni YA, Lee F, Koli K, Durrell M, Eavou R, Hanson H, Park SH, Duncan DT. Timmins L, et al. Soc Psychiatry Psychiatr Epidemiol. 2022 Oct;57(10):1999-2011. doi: 10.1007/s00127-022-02282-2. Epub 2022 Apr 22. Soc Psychiatry Psychiatr Epidemiol. 2022. PMID: 35460059 Free PMC article.
MeSH terms
LinkOut - more resources
Full Text Sources
Research Materials