Psychometric Analysis of MCQs Used in Assessing the Students at Entrance to a Medical College (original) (raw)
Related papers
Assessment of MCQs in MBBS program in college of Medicine, King Khalid University Abha
2017
Background Multiple choice questions represent one of the commonest methods of assessment in medical education. They believed to be reliable and efficient. Their quality depends on good item construction. Item analysis is used to assess their quality by computing difficulty index, discrimination index, distractor efficiency and test reliability.
Indian Journal of Community Medicine
Background: Multiple choice questions (MCQs) are frequently used to assess students in different educational streams for their objectivity and wide reach of coverage in less time. However, the MCQs to be used must be of quality which depends upon its difficulty index (DIF I), discrimination index (DI) and distracter efficiency (DE). Objective: To evaluate MCQs or items and develop a pool of valid items by assessing with DIF I, DI and DE and also to revise/ store or discard items based on obtained results. Settings: Study was conducted in a medical school of Ahmedabad. Materials and Methods: An internal examination in Community Medicine was conducted after 40 hours teaching during 1 st MBBS which was attended by 148 out of 150 students. Total 50 MCQs or items and 150 distractors were analyzed. Statistical Analysis: Data was entered and analyzed in MS Excel 2007 and simple proportions, mean, standard deviations, coefficient of variation were calculated and unpaired t test was applied. Results: Out of 50 items, 24 had "good to excellent" DIF I (31 -60%) and 15 had "good to excellent" DI (> 0.25). Mean DE was 88.6% considered as ideal/ acceptable and non functional distractors (NFD) were only 11.4%. Mean DI was 0.14. Poor DI (< 0.15) with negative DI in 10 items indicates poor preparedness of students and some issues with framing of at least some of the MCQs. Increased proportion of NFDs (incorrect alternatives selected by < 5% students) in an item decrease DE and makes it easier. There were 15 items with 17 NFDs, while rest items did not have any NFD with mean DE of 100%. Conclusion: Study emphasizes the selection of quality MCQs which truly assess the knowledge and are able to differentiate the students of different abilities in correct manner.
2015
Introduction: Multiple-choice questions (MCQs) are most widely used test format in health sciences today. The efficiency of MCQs as an efficient tool for evaluation solely rests upon theirquality which is best assessed by item and test analysis. Objectives:Toassess item and test quality and to explore the relationship between difficulty index (p-value) and discrimination indices (DI) with distractor efficiency (DE). Materials and Methods:The study was conducted among 40 fourth semester MBBS students in a medical college of Kolkata. Thirty MCQs administered in an internal examination in Community Medicine, were analysed for p-value, DI and DE. Reliability of the test was assessed by estimating the Kuder-Richardson 20 coefficient (KR20). Results:The mean score was 66.35 ± 17.29. Mean p value and DI were 61.92 ± 25.1% and 0.31 ± 0.27, respectively. DI was noted to be maximum at p value range between 40% and 60%.Combining the two indices, 14(46.67 %) items could be called 'ideal'...
Analysis of multiple choice questions (MCQ): important part of assessment of medical students
International Journal of Medical Research and Review, 2016
Background: Assessment influences students learning process that is why analysis of assessment allows us to conduct it properly and accurately. Tarrant M et al. and few others had done similar studies in past. Further studies are required to support the same and continuing awareness among teachers about better assessment of students. Methods: The observational, non-interventional and prospective study was carried out to analyse 100 MCQs used for assessment of 2 nd MBBS students. All MCQs were having single stem with four options, one being correct and other three distractors. Each MCQ was analyzed with three tools that were Difficulty Index (DIF I), Discrimination Index (DI) and Distracter Efficiency (DE). Chi square test was used for statistical analysis. Results: Total 74 out of 100 MCQs (74%) were recommended (30-70%) according to DIF I. According to DI, total 34 out of 100 MCQs were good (0.25-0.35) and 30 MCQs were excellent (>0.35). There were 18 non-functional (6%) distractors out of total 300. In none of the MCQs, all three were poor distractor. Association between difficulty index and discrimination index was statistically significant according chi-square test. (p value< 0.05) Conclusions: Properly constructed MCQs, according to these analysis tools, are best for student's assessment.
Pre and Post examination evaluation of MCQs and SEQs Items in Undergraduate MBBS mock examination
The Professional Medical Journal, 2020
Objectives: To evaluate the quality of MCQs and SEQs pre and post Mock examination of physiology and biochemistry and correlation between the scores of both at private medical college Lahore. Study Design: Descriptive study and convenient sampling was done. Setting: Fatima Memorial College of Medicine and Dentistry, Lahore. Period: September 2016 to September 2017. Material & Methods: 149 students in Physiology and 143 in biochemistry took Mock examination. 45 MCQs and 9 SEQs each in biochemistry and physiology were prepared according to the table of specification provided by University of Health Sciences (UHS) Lahore, Pakistan. Item assessment according to Blooms taxonomy was done and item flaws identified with cover test done for structural validity of the paper before the exam by two medical educationists. Item analysis with difficulty index, discrimination index and distraction efficiency were done post examination. Results: 84.4% in physiology and 51.1% in biochemistry were of ...
Study on item and test analysis of multiple choice questions amongst undergraduate medical students
International Journal Of Community Medicine And Public Health, 2017
Background: Item analysis is the process of collecting, summarizing and using information from students’ response to assess the quality of test items. However it is said that MCQs emphasize recall of factual information rather than conceptual understanding and interpretation of concepts. There is more to writing good MCQs than writing good questions. The objectives of the study was to assess the item and test quality of multiple choice questions and to deal with the learning difficulties of students, identify the low achievers in the test. Methods: The hundred MBBS students from Government medical college were examined. A test comprising of thirty MCQs was administered. All items were analysed for Difficulty Index, Discrimination Index and Distractor Efficiency. Data entered in MS Excel 2007 and SPSS 21 analysed with statistical test of significance. Results: Majority 80% items difficulty index is within acceptable range. 63% items showed excellent discrimination Index. Distractor e...
Pakistan Journal of Health Sciences
Impartiality and reliability of Multiple-Choice Questions (MCQs) render them an ideal assessment tool for measuring higher order cognition. Objective: To determine the perception of postgraduate medical trainees pertinent to the quality of MCQs administered in Mid Training Assessment (MTA) January 2022. Methods: A cross-sectional descriptive study was carried out among 40 postgraduate trainees who appeared in MTA during January 2022. About 23, 12 and 05 of these trainees were doing MS and MD training at Benazir Bhutto Hospital, Holy Family Hospital and DHQ Hospital Rawalpindi respectively. Trainees were enrolled in the study through consecutive non-probability sampling. The feedback from the trainees about quality of MCQs was gathered by means of a structured questionnaire based on 5-point likert scale. Data analysis was done by using SPSS version 25.0. Results: About 67% of the trainees were pleased with the quality and construct of MCQ in their assessment in terms of illustration ...
International Journal of Medical and Biomedical Studies
Introduction: Multiple Choice Questions (MCQs) is one of the most preferred tool of assessment in medical education as a part of formative as well as summative assessment. MCQ performance as an assessment tool can be statistically analysed by Item analysis. Thus, aim of this study is to assess the quality of MCQs by item analysis and identify the valid test items to be included in the question bank for further use. Materials and methods: Formative assessment of Ist MBBS students was carried out with 40 MCQs as a part of internal examination in Biochemistry. Item analysis was done by calculating Difficulty index (P), Discrimination index (d) and number of Non- functional distractors. Results: Difficulty index (P) of 65% (26) items was well within acceptable range, 7.5% (3) items were too difficult whereas 27.5% (11) items were in the category of too easy. Discrimination Index (d) of 70% (28) items fell in recommended category whereas 10% (4) items were with acceptable, and 20% (8) we...
Item analysis and validation of MCQs in paediatrics for final MBBS examinations
2014
Objectives: The study was designed to validate the MCQs used in Graduate examinations in Paediatrics using item analysis. Material and Methods: MCQs and their answers responded by 100 students in each batch appearing for their final MBBS examinations in paediatrics were analysed for 3 consecutive years from 2012 to 2014. Item analysis of each question were undertaken for difficulty index, discriminative index and discrimination effectiveness. Result: Wide variation in difficulty and discriminative indices were noted, often outside prescribed range, making many of these unsuitable for inclusion in the question bank. Conclusion: The teaching faculty, question setters and moderators need to be sensitized about its implication
What does item analysis tell us? Factors affecting the reliability of MCQs
Gezira Journal of Health Sciences
Introduction: Item analysis is a very important part of the examination cycle, as it can be used to judge the difficulty and discrimination of test items as well as their reliability. Objective: The objective of this study is to determine the relation between reliability and other components of the item analysis of Multiple Choice Question (MCQs). Methodology: The item analysis of 38 modules conducted at the Faculty of Medicine-Jazan University in the academic year 2010-2011 were further analyzed descriptively and correlation were made between the reliability co-efficient (KR-20) and other elements of the item analysis such as the difficulty and discrimination indices Results: All the items have five options, the percentage of nonfunctioning options is between 27.6% and 70% across items. The facility index is < 35% is found in 18.3% of the items and facility index of > 90% is a maximum of 75%. All exams have items with a discrimination index <19% .