The psychometric and pragmatic evidence rating scale (PAPERS) for measure development and evaluation - PubMed (original) (raw)
The psychometric and pragmatic evidence rating scale (PAPERS) for measure development and evaluation
Cara C Lewis et al. Implement Res Pract. 2021.
Abstract
To rigorously measure the implementation of evidence-based interventions, implementation science requires measures that have evidence of reliability and validity across different contexts and populations. Measures that can detect change over time and impact on outcomes of interest are most useful to implementers. Moreover, measures that fit the practical needs of implementers could be used to guide implementation outside of the research context. To address this need, our team developed a rating scale for implementation science measures that considers their psychometric and pragmatic properties and the evidence available. The Psychometric and Pragmatic Evidence Rating Scale (PAPERS) can be used in systematic reviews of measures, in measure development, and to select measures. PAPERS may move the field toward measures that inform robust research evaluations and practical implementation efforts.
Keywords: Measures; implementation; pragmatic; psychometric; rating scale.
© The Author(s) 2021.
Conflict of interest statement
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: The psychometric and pragmatic evidence rating scale (PAPERS) for measure development and evaluations. Funding for this study came from the National Institute of Mental Health, awarded to Dr. Cara C. Lewis as principal investigator (R01MH106510). Dr. Lewis is both an author of this manuscript and editor of the journal, Implementation Research and Practice. Due to this conflict, Dr. Lewis was not involved in the editorial or review process for this manuscript.
Similar articles
- Measuring readiness for implementation: A systematic review of measures' psychometric and pragmatic properties.
Weiner BJ, Mettert KD, Dorsey CN, Nolen EA, Stanick C, Powell BJ, Lewis CC. Weiner BJ, et al. Implement Res Pract. 2020 Aug 26;1:2633489520933896. doi: 10.1177/2633489520933896. eCollection 2020 Jan-Dec. Implement Res Pract. 2020. PMID: 37089124 Free PMC article. Review. - Pragmatic measures for implementation research: development of the Psychometric and Pragmatic Evidence Rating Scale (PAPERS).
Stanick CF, Halko HM, Nolen EA, Powell BJ, Dorsey CN, Mettert KD, Weiner BJ, Barwick M, Wolfenden L, Damschroder LJ, Lewis CC. Stanick CF, et al. Transl Behav Med. 2021 Feb 11;11(1):11-20. doi: 10.1093/tbm/ibz164. Transl Behav Med. 2021. PMID: 31747021 Free PMC article. - Measures of organizational culture, organizational climate, and implementation climate in behavioral health: A systematic review.
Powell BJ, Mettert KD, Dorsey CN, Weiner BJ, Stanick CF, Lengnick-Hall R, Ehrhart MG, Aarons GA, Barwick MA, Damschroder LJ, Lewis CC. Powell BJ, et al. Implement Res Pract. 2021 Jun 21;2:26334895211018862. doi: 10.1177/26334895211018862. eCollection 2021 Jan-Dec. Implement Res Pract. 2021. PMID: 37090009 Free PMC article. Review. - Measuring characteristics of individuals: An updated systematic review of instruments' psychometric properties.
Stanick C, Halko H, Mettert K, Dorsey C, Moullin J, Weiner B, Powell B, Lewis CC. Stanick C, et al. Implement Res Pract. 2021 Mar 23;2:26334895211000458. doi: 10.1177/26334895211000458. eCollection 2021 Jan-Dec. Implement Res Pract. 2021. PMID: 37090010 Free PMC article. Review.
Cited by
- Application of the PAPERS Grading Criteria Within a Rapid Evidence Review to Determine the Psychometric and Pragmatic Properties of Patient Empowerment Tools.
Woolley KE, Jones NJ, Rahim A, Withers KL, Letchford R. Woolley KE, et al. J Patient Exp. 2024 Aug 21;11:23743735241272191. doi: 10.1177/23743735241272191. eCollection 2024. J Patient Exp. 2024. PMID: 39176305 Free PMC article. - Guideline for reporting systematic reviews of outcome measurement instruments (OMIs): PRISMA-COSMIN for OMIs 2024.
Elsman EBM, Mokkink LB, Terwee CB, Beaton D, Gagnier JJ, Tricco AC, Baba A, Butcher NJ, Smith M, Hofstetter C, Aiyegbusi OL, Berardi A, Farmer J, Haywood KL, Krause KR, Markham S, Mayo-Wilson E, Mehdipour A, Ricketts J, Szatmari P, Touma Z, Moher D, Offringa M. Elsman EBM, et al. Qual Life Res. 2024 Aug;33(8):2029-2046. doi: 10.1007/s11136-024-03634-y. Epub 2024 Jul 9. Qual Life Res. 2024. PMID: 38980635 Free PMC article. - Reliability and validity of intervention characteristic measures for assessing barriers to evidence-based practice use.
Jones SMW, Weiner BJ. Jones SMW, et al. Transl Behav Med. 2024 Apr 29;14(5):304-309. doi: 10.1093/tbm/ibae013. Transl Behav Med. 2024. PMID: 38518799 - The Case for Assessing and Reporting on Facilitator Fidelity: Introducing the Fidelity of Implementation in Parenting Programs Guideline.
Martin M, Shenderovich Y, Caron EB, Smith JD, Siu G, Breitenstein SM. Martin M, et al. Glob Implement Res Appl. 2024;4(1):1-10. doi: 10.1007/s43477-023-00092-5. Epub 2023 Sep 9. Glob Implement Res Appl. 2024. PMID: 38371716 Free PMC article. - What do you think it means? Using cognitive interviewing to improve measurement in implementation science: description and case example.
Patel-Syed Z, Becker S, Olson M, Rinella H, Scott K. Patel-Syed Z, et al. Implement Sci Commun. 2024 Feb 14;5(1):14. doi: 10.1186/s43058-024-00549-0. Implement Sci Commun. 2024. PMID: 38355677 Free PMC article.
References
- Aarons G. A., Green A. E., Trott E., Willging C. E., Torres E. M., Ehrhart M. G., Roesch S. C. (2016). The roles of system and organizational leadership in system-wide evidence-based intervention sustainment: A mixed-method study. Administration and Policy in Mental Health and Mental Health Services Research, 43(6), 991–1008. 10.1007/s10488-016-0751-4 - DOI - PMC - PubMed
- Allen P., Pilar M., Walsh-Bailey C., Hooley C., Mazzucca S., Lewis C. C., Mettert K. D., Dorsey C. N., Purtle J., Kepper M. M., Baumann A. A., Brownson R. C. (2020). Quantitative measures of health policy implementation determinants and outcomes: A systematic review. Implementation Science, 15(1), 47. 10.1186/s13012-020-01007-w - DOI - PMC - PubMed
- Farahnak L. R., Ehrhart M. G., Torres E. M., Aarons G. A. (2020). The influence of transformational leadership and leader attitudes on subordinate attitudes and implementation success. Journal of Leadership & Organizational Studies, 27(1), 98–111. 10.1177/1548051818824529 - DOI
- McLoughlin G. M., Allen P., Walsh-Bailey C., Brownson R. C. (2021). A systematic review of school health policy measurement tools: Implementation determinants and outcomes. Implementation Science Communications, 2(1), 67. 10.1186/s43058-021-00169-y. PMID: 34174969; PMCID: PMC8235584. - DOI - PMC - PubMed
LinkOut - more resources
Full Text Sources