Inter-Rater Reliability Of Objective Structured Long Examination Record (original) (raw)
Related papers
Practical examinations play a very important role in the certification of candidates before they can practice medicine. Long case presentation has been a method of assessment of medical students in practical examination for a long time in major undergraduate and postgraduate subjects. This system of evaluation is subjective, broad based and often affected by subject and environmental factors. This leads to inconsistencies in grading. Objective structured long examination record (OSLER) is a more structured and objective method of assessment. 1 We thus introduced this concept in assessment of the postgraduate students in anaesthesiology. METHODS The pilot project on OSLER was undertaken between the year 2009-2010 with the permission of the institutional dean. The method of assessment was conducted on eight, third year postgraduate students in anaesthesiology. The post graduate program for long case discussions was planned. The evaluation of each case discussion was structured under ten broad headings and further sub-units as per the requirement of the subject.
Academia Journal of Medicine, 2019
Background: Most medical colleges in India traditionally use the Long Case Examination (LCE) in both formative and summative patterns of assessment in clinic. This age-old issue has been addressed over the years by the introduction of several modifications of the LCE such as OSCE and OSLER. OSCE, though attractive in its brevity and specificity of focus, falls short in terms of being comprehensive. This study was taken up to compare conventional LCE to OSCE and OSLER. Subjects and Methods: Final year MBBS students, posted to Department of General Medicine, HIMS, Hassan were studied for 5 months. Type of study: Prospective observational study Four examiners were selected for the formative assessment: Examiners 1 and 2 were asked to assess the students assigned to Group 1; Examiners 3 and 4 were asked to assess Group 2. The OSLER method concentrates on the following ten items: (1) pace and clarity of presentation, (2) communication process, (3) systematic approach, (4) Establishment of case facts, (5) systematic examination, (6) Examination technique, (7) Establishment of correct physical findings. Results: The mean time taken for the assessment in Group 1 was 28.22 minutes and the SD was 7.588. The mean time taken for the assessment in Group 2 was 17.09 minutes and the SD was 4.546. Conclusion: In our study, the students seem to prefer the time-tested long case examination and did score better marks with the conventional method than with OSLER.
Background/Purpose: one of the most important mission and vision for University of Dammam is to send skilled physicians to deal and treat the patients in professional manner, which cannot be without perfect evaluation of the knowledge and clinical skills of the undergraduates. The Objective Structured Clinical Examination (OSCE) was introduced to increase the reliability and validity of the exams. Method: A pilot study was conducted over one semester. A total of 299 examinees (4th year = 207 and 5th year = 92) took the OSCE and written exams in three groups. The OSCE consisted of 18–20 clinical stations, which included taking histories, physical examinations, communication skills, and data interpretation. The written exam contained 80 multiple-choice questions. Result: Cronbach's alphas by group were 0.7, 0.8, and 0.9 (4th year) and 0.62, 0.79, and 0.85 (5th year). Correlations for all stations ranged from 0.6 to 0.8, which indicated good stability and internal consistency. The reliability of the written exam was 0.79 (4th year) and 0.85 (5th year).The validity of the OSCE was assessed by Pearson's correlations, which were 0.63 (4th year), 0.6 (5th year), and 0.66 (overall). Conclusion: The OSCE is a perfect tool for undergraduates assessment because of better reliability and validity which can be measured .
2012
effectively and perform safely in the context of 7 periods . Differences between self-and teacherassessment can lead to productive teacher-student SHORT COMMUNICATION ABSTRACT Objective: To compare students' self-assessment with assessment by examiners in a formative observed structured clinical examination (OSCE) and comparison between cumulative score and global rating scale for students and faculty evaluation. th Methodology: This cross sectional study was carried out on 6 year medical students of Oman Medical College, Family Medicine rotation in a formative OSCE. A self-assessment questionnaire was completed by 15, Year 6 students immediately after the OSCE. Statistical analysis was done with SPSS 16.0 for Windows. Regression analysis was used to correlate self reported rating and actual performance in each station. A p value of < 0.05 was considered significant. Correlation between cumulative score and global rating scale for students and faculty evaluation was calculated by Pearson correlation.
Enhancing the defensibility of examiners\u27 marks in high stake OSCEs
2018
Background: Most assessments in health professions education consist of knowledge-based examinations as well as practical and clinical examinations. Among the most challenging aspects of clinical assessments is decision making related to borderline grades assigned by examiners. Borderline grades are commonly used by examiners when they do not have sufficient information to make clear pass/fail decisions. The interpretation of these borderline grades is rarely discussed in the literature. This study reports the application of the Objective Borderline Method (version 2, henceforth: OBM2) to a high stakes Objective Structured Clinical Examination undertaken at the end of the final year of a Medicine program in Australia. Methods: The OBM2 uses all examination data to reclassify borderline grades as either pass or fail. Factor analysis was used to estimate the suitability of data for application of OBM2. Student\u27s t-tests, utilising bootstrapping, were used to compare the OBM2 with \...
Utility of OSLER for assessing enhancement of learning in postgraduate students
South-East Asian Journal of Medical Education, 2020
Background: Assessing the clinical acumen of the medical students both at undergraduate and postgraduate levels is a challenge faced by the medical teachers worldwide. For years knowledge and skills were being assessed using traditional long cases and short cases. Structuring the long case, like Objective Structured Long Examination Record (OSLER) can help improve the reliability of the long case. Method: A six month interventional study was conducted in Department of Obstetrics & Gynaecology, JNMC, Wardha. It included, 12 first year and 12 second year postgraduate students, who were subjected to two OSLER interventions at interval of one month giving a total sample size of 48 OSLER assessments. Result: It was observed that the scores obtained by the students, had improved significantly in the 2nd OSLER intervention as compared to the 1st OSLER intervention. For the first year postgraduate students the mean scores were 48.33 in the 1st encounter as compared to 60.83 in the 2nd encounter. For the second year postgraduate students, the means scores were 53.75 in the 1st encounter and 65.41 in the 2nd encounter. The 'P' value was found to be 0.0001 which was statically significant. Conclusion: OSLER can be used effectively for testing and for assessing the enhancement of learning and clinical acumen of postgraduate students. Interventions done regularly can go a long way in improving the clinical skills of the students.
Enhancing the defensibility of examiners' marks in high stake OSCEs
BMC medical education, 2018
Most assessments in health professions education consist of knowledge-based examinations as well as practical and clinical examinations. Among the most challenging aspects of clinical assessments is decision making related to borderline grades assigned by examiners. Borderline grades are commonly used by examiners when they do not have sufficient information to make clear pass/fail decisions. The interpretation of these borderline grades is rarely discussed in the literature. This study reports the application of the Objective Borderline Method (version 2, henceforth: OBM2) to a high stakes Objective Structured Clinical Examination undertaken at the end of the final year of a Medicine program in Australia. The OBM2 uses all examination data to reclassify borderline grades as either pass or fail. Factor analysis was used to estimate the suitability of data for application of OBM2. Student's t-tests, utilising bootstrapping, were used to compare the OBM2 with 'traditional'...
PhD Thesis, 2019
QUALITY IMPROVEMENT FRAMEWORK DESIGN OF OBJECTIVE STRUCTURED CLINICAL EXAMINATION IN HAWLER MEDICAL UNIVERSITY. By Nazdar Al-Khateeb. Background and objectives: An Objective Structured Clinical Examination (OSCE) is a common measurement tool used to assess clinical competencies of the medical students. In order to draw meaningful interpretations of the assessment results, the OSCE needs to be valid and reliable. This study aims at further improvement of OSCE as an assessment tool for undergraduate medical education in Hawler Medical University. Methods: This study was conducted in Hawler Medical University, College of Medicine from June 2016 to November 2019. A modified action research approach was used to explore a number of OSCE characteristics at Hawler Medical University: the first part included a study on the impact of formative OSCE on performance of 5th year students in their summative OSCE. The second part explored the perception of 81 examiners from clinical departments on the OSCE, once this was achieved, a standardized OSCE was designed for clinical departments. This was followed by evaluating the OSCE scores of 150 sixth year students to ensure OSCE’s validity and reliability. The cut score of the OSCE were determined by different standard setting methods, and a defensible pass/fail decision was developed through novel competency method. Finally, a sample of 196 students shared their perception on OSCE. Factor analysis method was used to estimate latent traits that could not be measured by a single survey question. Results: The randomized controlled trial revealed that single formative-OSCE did not improve the students’ performance in a subsequent summative-OSCE. On the other hand, the reliability and other psychometric indices of the final OSCE were in acceptable range. A novel competency method was suggested to reflect the desired student ability for each competency. This study has provided a clear understanding of the strengths and weaknesses of the original OSCE from both examiners and students' perspectives that could add insight to more reform. Conclusions: Standardizing OSCE and doing it in a single day further improve the outcomes of the exam. Some practical suggestions to increase the reliability of OSCEs and how to collect validity evidence on OSCE were presented. OSCE results can give a meaningful feedback to students on their competencies. By Nazdar Al-Khateeb. Keywords: OSCE, Clinical competence, Standard setting, Educational measurement, Psychometrics.
Journal of Postgraduate Medical Institute, 2013
effectively and perform safely in the context of 7 periods . Differences between self-and teacherassessment can lead to productive teacher-student SHORT COMMUNICATION ABSTRACT Objective: To compare students' self-assessment with assessment by examiners in a formative observed structured clinical examination (OSCE) and comparison between cumulative score and global rating scale for students and faculty evaluation. th Methodology: This cross sectional study was carried out on 6 year medical students of Oman Medical College, Family Medicine rotation in a formative OSCE. A self-assessment questionnaire was completed by 15, Year 6 students immediately after the OSCE. Statistical analysis was done with SPSS 16.0 for Windows. Regression analysis was used to correlate self reported rating and actual performance in each station. A p value of < 0.05 was considered significant. Correlation between cumulative score and global rating scale for students and faculty evaluation was calculated by Pearson correlation.
Sudan Journal of Medical Sciences
Introduction: Students' perception of an examination reflects their feelings, while its item analysis refers to a statistical analysis of students’ responses to examination items. The study was conducted to compare the student’s perception towards examination to its item analysis. Material and methods: This is a cross-sectional study conducted at the college of medicine, from January to April 2019. The study used a structured questionnaire and standardized item analysis of students’ examinations. All students who had registered for semester two (2018-2019) were included in the study. Exclusion criteria included students who refused to participate in the study or those who did not fill the questionnaire. Result: KR-20 of the examination was 0.906. The average difficulty index of the examination was 69.4. The response rate of the questionnaire was 88.9% (40/45). Students considered the examination to be easy (70.4%). A significant correlation was reported between student percept...