Comparison between students’ perception and examination validity, reliability and items difficulty: a cross-sectional study (original) (raw)
Related papers
2020
Introduction: While student’s perception of an exam is a reflection of their feelings toward the exmination items, item analysis is a statistical analysis of their responses to examination items. The study aims to compare students’ perception toward the difficulty of an examination with the results of item analysis and examination reliability and validity. Materials and methods: This is a cross-sectional study conducted in the College of Medicine between January and April 2019. The study uses a structured questionnaire and standardized item analysis of students’ examination. Results: Overall, 80 items were analyzed in this study. Kuder–Richardson Formula 20 of the examination was 0.906. The average difficulty index of the examination items was 69.4 (± 21.86). The response rate of the questionnaire was 88.9% (40/45). Students considered the examination as easy (70.4%). Students’ perception toward the difficulty of the individual items shows a moderate positive correlation between eas...
IntroductionStudent perception of an exam is a reflection of their feelings towards the exam items, while item analysis is a statistical analysis of students’ responses to exam items. The study was formulated to compare the student’s perception of the results of item analysis.Material and methodsType of the study is cross-sectional. The study was conducted in the college of medicine, in the duration from January to April 2019. The study uses a structured questionnaire and standardized item analysis of students’ exam. Participants are students registered for semester two level year (2018-2019). Exclusion criteria included all students who refused to participate in the study or do not fill the questionnaire.ResultThe response rate of the questionnaire was 88.9% (40/45). Students considered the exam as easy (70.4%). The average difficulty index of the exam is acceptable. KR-20 of the exam was 0.906. A significant correlation was reported between student perceptions towards exam difficu...
Nigerian journal of clinical practice
Item analysis is an effective method in the evaluation of multiple-choice achievement tests. This study aimed to compare the classical and the latent class models used in item analysis, as well as their efficacy in the evaluation of the examinations of the medical faculty. The achievement tests in the medical faculty were evaluated using different methods. The two methods used were the classical and the latent class models. Among the classical methods, Cronbach's alpha, split half methods, item discrimination, and item difficulty was investigated. On the other hand, various models of item response theory (IRT) and their statistics were compared in the group of latent class methods. Reliability statistics had values above 0.87. Item no. 7 was found easy, item no. 45 difficult and item no. 64 fairly difficult according to the evaluations done by classical and item response theories. In terms of item discrimination, item no. 45 had lower, item no. 7 had middle and item no. 64 had h...
Item analysis of multiple choice questions: Assessing an assessment tool in medical students
Aim: Assessment is a very important component of the medical course curriculum. Item analysis is the process of collecting, summarizing, and using information from student's responses to assess the quality of multiple-choice questions (MCQs). Difficulty index (P) and discrimination index (D) are the parameters used to evaluate the standard of MCQs. The aim of the study was to assess quality of MCQs. Materials and Methods: The study was conducted in the Department of Pathology. One hundred and twenty, 2nd year MBBS students took the MCQs test comprising 40 questions. There was no negative marking and evaluation was done out of 40 marks, and 50% score was the passing mark. Postvalidation of the paper was done by item analysis. Each item was analyzed for difficulty index, discrimination index, and distractor effectiveness. The relationship between them for each item was determined by Pearson correlation analysis using SPSS 20.0. Results: Difficulty index of 34 (85%) items was in the acceptable range (P = 30–70%), 2 (5%) item was too easy (P >70%), and 4 (10%) items were too difficult (P <30%). Discrimination index of 24 (60%) items was excellent (D >0.4), 4 (10%) items were good (D =0.3–0.39), 6 (15%) items were acceptable (D =0.2–0.29), and 6 (15%) items were poor (D < 0–0.19). A total 40 items had 120 distractors. Amongst these, 6 (5%) were nonfunctional distracters, 114 (95%) were functional distracters. The discrimination index exhibited positive correlation with difficulty index (r = 0.563, P = 0.010, significant at 0.01 level [two-tailed]). The maximum discrimination (D = 0.5–0.6) was observed in acceptable range (P = 30–70%). Conclusion: In this study, the majority of items fulfilled the criteria of acceptable difficulty and good discrimination. Moderately easy/difficult had the maximal discriminative ability. Very difficult item displayed poor discrimination, but the very easy item had high discrimination index, indicating a faulty item, or incorrect keys. The results of this study would initiate a change in the way MCQ test items are selected for any examination, and there should be proper assessment strategy as part of the curriculum development.
South-East Asian Journal of Medical Education
Objective: To find out difficulty index (P) and discrimination index (D) of the items-Multiple True or False (MTF) questions-set for the examinations conducted by constituent departments of a medical college, namely: anatomy, physiology, biochemistry, pathology, pharmacology and microbiology. Methods: Scores obtained by students of first year MBBS (March 2010 batch) in anatomy, physiology, biochemistry and second year students (March 2009 batch) in pathology, pharmacology and microbiology were taken. MTF component of the examination of block 1, 2, 3 and 4 were considered. Difficulty index (P) and discrimination index (D) of MTF component of block 1, 2, 3 and 4 were analyzed using Microsoft excel. Correlations of MTF and essay scores of the examinations were also performed. Results: The average easy questions(P ≥ 75%) were 48%, 65% and 51% & percentage of average difficult questions (P ≤ 25%) were 3%, 1% and 2% in anatomy, physiology and biochemistry respectively. Anatomy, physiology and biochemistry had 71%, 56% and 67% of questions with discrimination index D ≥ 0.2. The average easy questions were 64%, 51% and 46% & percentage of average difficult questions was 3%, 3% & 4% in pathology, pharmacology and microbiology respectively. Pathology, pharmacology and microbiology had 40%, 51% and 50% of questions with discrimination index D ≥ 0.2. The essay and MTF scores of each subject showed strong and significant correlations. Conclusion: The difficulty index obtained from the analysis shows that the questions were easy for the students. However, the study shows that about 50% of the questions were capable of discriminating students with higher ability from those with lower ability (D ≥ 0.2).
International Journal of Research in Medical Sciences, 2016
Background: Multiple choice questions (MCQs) are usually used to assess students in different educational streams. However, the MCQs to be used should be of quality which depends upon its difficulty index (DIF I), discrimination index (DI) and number of Non-functional distracter (NFD). Objective of the study is to evaluate the quality of MCQs, for creating a valid question bank for future use and to identify the low achievers, whose problems can be corrected by counselling or modifying learning methods. This study was done in Kalinga Institute of Medical Science (KIMS) Bhubaneswar. Methods: A part completion test in the department of pediatrics was done. Total 25 MCQs and 75 distracters were analyzed. Item analysis was done for DIF I and DI and presence of number of NFD. Results: Difficulty index of 14 (56%) items was in the acceptable range (p value 30-70%), 8 (32%) items were too easy (p value >70%) and 2 (8%) items were too difficult (p value <30%). Discrimination index of 12 (48%) items was excellent (d value>0.35), 3 (12%) items was good (d value 0.20-0.34) and 8(32%) items were poor (d value<0.2%). Out of 75 distracters, 40 (53.4%) NFDs were present in 22 items. 3 (12%) items had no NFDs, whereas 8 (32%), 10 (40%), and 4 (16%) items contained 1, 2, and 3 NFD respectively. Conclusion: Item analysis is a simple and feasible method of assessing valid MCQs in order to achieve the ultimate goal of medical education.
International journal of basic and clinical pharmacology, 2018
Background: Multiple choice questions (MCQs) are a common method of assessment of medical students. The quality of MCQs is determined by three parameters such as difficulty index (DIF I), discrimination index (DI), and Distractor efficiency (DE). Item analysis is a valuable yet relatively simple procedure, performed after the examination that provides information regarding the reliability and validity of a test item. The objective of this study was to perform an item analysis of MCQs for testing their validity parameters. Methods: 50 items consisting of 150 distractors were selected from the formative exams. A correct response to an item was awarded one mark with no negative marking for incorrect response. Each item was analysed for three parameters such as DIF I, DI, and DE. Results: A total of 50 items consisting of 150 Distractor s were analysed. DIF I of 31 (62%) items were in the acceptable range (DIF I= 30-70%) and 30 had ‘good to excellent’ (DI >0.25). 10 (20%) items were ...
Evaluation of Multiple Choice Questions by Item Analysis in a Medical College- A Pilot Study
journal of medical science and clinical research, 2017
Introduction: Assessment is an integral part of any learning and training. Multiple choice questions (MCQs) are a widely used tool in assessment protocols. To increase the validity of MCQs standard prevalidation and post validation protocols are recommended. Item Analysis is a post validation procedure. Aims and Objectives: Difficulty index, Discrimination index and Distracter effectiveness are the parameters used to evaluate the standard of MCQs Materials and Methods: This study was conducted in the Department of Pathology ata Medical College. This is a retrospective study. The Term end examination MCQ paper after the 1 st semester was assessed. Based on the answers marked by the students the Difficulty index, Discrimination index and Distracter effectiveness were calculated. Results: In the present study, according to the Difficulty index criteria 50% of the MCQs were acceptable, of which 15% were ideal. On the basis of Discrimination index, 60% were good discriminator and 35% of the MCQs were excellent with a DI greater than 0.35%. In 45% MCQs the distracters were effective. Only 7 out of 20 MCQs satisfied all the criteria for an ideal MCQ. Conclusion: This exercise was an eye opener revealing the quality of the MCQs and it will also help while formulating MCQs for future exams.
Journal of Chitwan Medical College
Background: Objective Structured Clinical Examination (OSCE) has been globally recognized as gold standard of performance based assessment that aids in evaluating the clinical competency of medical undergraduate students. Hence, it increases the confidence of students in their learning ability by helping them to identify areas of weakness and gaps in their competencies. This study was conducted to identify perception towards OSCE among medical students. Methods: A descriptive cross-sectional study design was adopted and a total 173 respondents were selected from MBBS 4th and 5th year of Chitwan Medical College by using probability stratified proportionate random sampling technique. Self-administered questionnaire with 5-point likert scale were used to collect the data. Descriptive and inferential statistics was used to analyze data. Results: About half of the Bachelor of Medicine and Bachelor of Surgery (MBBS) students had positive perception toward OSCE. There is significant associ...
Students Preference for Various Types of Assessment in Anatomy Examination. Impact factor 0.471
Background: The use of variety of different assessment methods has been characteristics of medical education. The aims of this study were to determine the students preference for various types of assessments that form the second MB, BS examination in Anatomy and to know if there is any correlation between these methods. Methods: Performance of 242 students in second MB, BS examination was considered for this study. Four difference assessments were employed; The Essay consists of 5 questions of 20 marks each, Multiple choice questions consist of 100 questions of one mark each, Practical examinations has 50 stations, students rotate from specimen to specimen and are given one minute to answer each question. Oral is directed by the external examiner. The result obtained was analyzed using SPSS version 17. Results: The mean scores for MCQ, Practical, Essay and Oral were 40%, 48%, 47% and 51% respectively. Those who scored <50% in MCQs, practical, essay and oral were; 81.41%, 44.63%, 59.92% and 26.86% respectively. It showed that the students scored maximum mark (80%) in oral followed by essay (74%), practical (72%) and MCQ (68%) in that order. Lowest marks were scored in oral and practical (10%).There was strong positive correlations among the various types of assessments. Conclusion: The students performed best in oral followed by practical, essay and MCQs in that order. The four components of the assessment correlated strongly. This combination of variety of assessment help to minimize the short comings of each one of them and help to bring objectivity to the grading system. Hence multidimensional assessment helps to evaluate the various skills of the students and the objectives of medical education and produce physicians that are both skilled and knowledgeable.