Creating Evidence to Advocate the Validity of Results of Clinical Performance in the Undergraduate Surgery Clerkship (original) (raw)

Background/ Purpose: Evidential bases were not performed en masse to validate assessment results in the undergraduate Surgery clerkship in King Abdulaziz University (KAU). This study aimed at producing a comprehensive package of evidence to prove validity of students' clinical performance assessment results (as defined by Messick's framework). Method: Guided by Messick's conceptual framework, the problem was analyzed. Hands-on faculty development on creating an exam blueprint was done: 1. Learning objectives (LOs) revised; 2. Alignment secured; 3. Weight of (LOs) determined; 4. Number of items/topic/domain calculated; and 5. Appropriate assessment methods selected..Quantitative evidences as reliability and correlation coefficients of various validity components were calculated. The underlying values that scaffold validity evidences were explored via a Focus Group Discussion and the results analyzed by content analysis. Results: 1. The weight of different domains in the test equally reflected their weight in the curriculum (content validity); 2. Positive unintended consequences resulted from the new assessment approach (consequential validity); 3. There was a statistically significant correlation among various assessment methods that provided evidence for concurrent and predictive validity; 4. Success rates and grades distribution alone could not provide evidence to advocate an argument on validity of results. Conclusion: A newly introduced assessment plan with new tools had to be validated by pursuing a comprehensive, unified approach to create evidence from multiple sources of data in order to support the argument of advocating the assessment results.