A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research - PubMed (original) (raw)
A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research
Terry K Koo et al. J Chiropr Med. 2016 Jun.
Erratum in
- Erratum to "A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research" [J Chiropr Med 2016;15(2):155-163].
[No authors listed] [No authors listed] J Chiropr Med. 2017 Dec;16(4):346. doi: 10.1016/j.jcm.2017.10.001. Epub 2017 Nov 9. J Chiropr Med. 2017. PMID: 29276468 Free PMC article.
Abstract
Objective: Intraclass correlation coefficient (ICC) is a widely used reliability index in test-retest, intrarater, and interrater reliability analyses. This article introduces the basic concept of ICC in the content of reliability analysis.
Discussion for researchers: There are 10 forms of ICCs. Because each form involves distinct assumptions in their calculation and will lead to different interpretations, researchers should explicitly specify the ICC form they used in their calculation. A thorough review of the research design is needed in selecting the appropriate form of ICC to evaluate reliability. The best practice of reporting ICC should include software information, "model," "type," and "definition" selections.
Discussion for readers: When coming across an article that includes ICC, readers should first check whether information about the ICC form has been reported and if an appropriate ICC form was used. Based on the 95% confident interval of the ICC estimate, values less than 0.5, between 0.5 and 0.75, between 0.75 and 0.9, and greater than 0.90 are indicative of poor, moderate, good, and excellent reliability, respectively.
Conclusion: This article provides a practical guideline for clinical researchers to choose the correct form of ICC and suggests the best practice of reporting ICC parameters in scientific publications. This article also gives readers an appreciation for what to look for when coming across ICC while reading an article.
Keywords: Reliability and validity; Research; Statistics.
Figures
Fig 1
A flowchart showing the selection process of the ICC form based on the experimental design of a reliability study. The process involves the selection of the appropriate model (ie, 1-way random effects, 2-way random effects, or 2-way fixed effects), type (ie, single rater/measurement or the mean of k raters/measurements), and definition of relationship considered to be important (ie, consistency or absolute agreement).
Fig 2
Hypothetical data illustrating how different forms of ICC can give different results when applied to the same set of data and how the nature of the data affects the ICC estimates of different forms.
Fig 3
A flowchart showing readers how to interpret ICC in published studies. Values less than 0.5 are indicative of poor reliability, values between 0.5 and 0.75 indicate moderate reliability, values between 0.75 and 0.9 indicate good reliability, and values greater than 0.90 indicate excellent reliability.
Similar articles
- Development and Reliability Evaluation of the Movement Rating Instrument for Virtual Reality Video Game Play.
Levac D, Nawrotek J, Deschenes E, Giguere T, Serafin J, Bilodeau M, Sveistrup H. Levac D, et al. JMIR Serious Games. 2016 Jun 1;4(1):e9. doi: 10.2196/games.5528. JMIR Serious Games. 2016. PMID: 27251029 Free PMC article. - Intraclass correlations as estimates of interrater reliability in nursing research.
Laschinger HK. Laschinger HK. West J Nurs Res. 1992 Apr;14(2):246-51. doi: 10.1177/019394599201400213. West J Nurs Res. 1992. PMID: 1561790 - Updated guidelines on selecting an intraclass correlation coefficient for interrater reliability, with applications to incomplete observational designs.
Ten Hove D, Jorgensen TD, van der Ark LA. Ten Hove D, et al. Psychol Methods. 2022 Sep 1. doi: 10.1037/met0000516. Online ahead of print. Psychol Methods. 2022. PMID: 36048052 - Evaluating test-retest reliability in patient-reported outcome measures for older people: A systematic review.
Park MS, Kang KJ, Jang SJ, Lee JY, Chang SJ. Park MS, et al. Int J Nurs Stud. 2018 Mar;79:58-69. doi: 10.1016/j.ijnurstu.2017.11.003. Epub 2017 Nov 8. Int J Nurs Stud. 2018. PMID: 29178977 Review. - The interrater and intrarater reliability of the functional movement screen: A systematic review with meta-analysis.
Cuchna JW, Hoch MC, Hoch JM. Cuchna JW, et al. Phys Ther Sport. 2016 May;19:57-65. doi: 10.1016/j.ptsp.2015.12.002. Epub 2015 Dec 18. Phys Ther Sport. 2016. PMID: 26777566 Review.
Cited by
- Development of low-cost pressure mapping device to evaluate force distribution for seat cushion modification.
Jarumethitanont W, Manupibul U, Tanthuwapathom R, Prasertsukdee S, Limroongreungrat W, Charoensuk W. Jarumethitanont W, et al. Sci Rep. 2024 Sep 18;14(1):21804. doi: 10.1038/s41598-024-72471-3. Sci Rep. 2024. PMID: 39294267 - Assessment of verbal memory in Parkinson's disease utilizing a virtual reality-based Rey Auditory Verbal Learning Test.
Gottlieb A, Kimel-Naor S, Zeilig G, Schnaider Beeri M, Plotnik M. Gottlieb A, et al. Sci Rep. 2024 Sep 18;14(1):21792. doi: 10.1038/s41598-024-71618-6. Sci Rep. 2024. PMID: 39294213 - Impaired Social Attention and Cognitive Empathy in a Paediatric Sample of Children with Symptoms of Anxiety.
Eaton S, Dorrans EM, van Goozen SHM. Eaton S, et al. Res Child Adolesc Psychopathol. 2024 Sep 18. doi: 10.1007/s10802-024-01240-7. Online ahead of print. Res Child Adolesc Psychopathol. 2024. PMID: 39292383 - The German version of the Pregnancy Physical Activity Questionnaire: a translation, cross-cultural adaptation, reliability and validity assessment.
Spiller M, Ferrari N, Joisten C. Spiller M, et al. BMC Pregnancy Childbirth. 2024 Sep 17;24(1):604. doi: 10.1186/s12884-024-06804-5. BMC Pregnancy Childbirth. 2024. PMID: 39289611 Free PMC article. - Preoperative Workup of Operative Hip Fracture Patients: A Survey.
Esper GW, Anil U, Cavaleri SG, Furgiuele DL, Zaretsky J, Konda SR, Egol KA. Esper GW, et al. HSS J. 2024 May;20(2):237-244. doi: 10.1177/15563316231158546. Epub 2023 Mar 9. HSS J. 2024. PMID: 39281995
References
- Daly LE, Bourke GJ. Blackwell Science Ltd; Oxford: 2000. Interpretation and use of medical statistics.
- Portney LG, Watkins MP. Prentice Hall; New Jersey: 2000. Foundations of clinical research: applications to practice.
- Bruton A, Conway JH, Holgate ST. Reliability: what is it, and how is it measured? Physiotherapy. 2000;86:94–99.
- Ebel RL. Estimation of the reliabilty of ratings. Psychometrika. 1951;16:407–424.
- Bartko JJ. The intraclass correlation coefficient as a measure of reliability. Psychol Rep. 1966;19:3–11. - PubMed
LinkOut - more resources
Full Text Sources
Other Literature Sources