Comparison of Oral Examination and Examination Methods Based on Multiple-Choice Questions Using Personal Computers (original) (raw)
Related papers
Computers & Education, 2010
The aim of the present research was to compare the use of multiple-choice questions (MCQs) as an examination method, to the examination based on constructed-response questions (CRQs). Despite that MCQs have an advantage concerning objectivity in the grading process and speed in production of results, they also introduce an error in the final formulation of the score. The error is traced to the probability of answering a question by chance or based on an instinctive feeling, which does not enable the ascertainment of the knowledge of the whole background included in the question. In the present study, both MCQ and CRQ tests were given to examinees, in the framework of a computer-based learning system. Avoiding the procedure of mixed scoring, e.g. both positive and negative markings, a set of pairs of MCQs was composed. The MCQs in each pair were similar concerning the same topic, but this similarity was not evident for an examinee that did not possess adequate knowledge on the topic addressed in the questions of the pair. The examination based on these ''paired" MCQs, by using a suitable scoring rule, when made to the same sample of students, on the same topics and with the same levels of difficulty, gave results that were statistically indistinguishable with the grades produced by an examination based on CRQs, while both the ''paired" MCQ test results and the CRQ test results differed significantly from those obtained from a MCQ test using positive-only scoring rule.
Comparison of oral examination and electronic examination using paired multiple-choice questions
Computers & Education, 2011
The aim of the present research was to compare the use of multiple-choice questions (MCQs) as an examination method against the oral examination (OE) method. MCQs are widely used and their importance seems likely to grow, due to their inherent suitability for electronic assessment. However, MCQs are influenced by the tendency of examinees to guess answers, warranting research concerning scoring rules different from the simple positive-grades-only scores rule. Alternatively, OE is used in tertiary education, since it enables the assessment of intellectual capabilities and personal traits to a level not found in most other examination formats. However, the significant resource requirements of OE, especially in structured forms, might excessively strain the resources of academic institutions. In the present study, an MCQ test was given to examinees, in the framework of a computer-based learning system. The same examinees took also an OE possessing elements of structure, with three examiners concurrently and independently grading each of the examinees. In the MCQs examination a set of pairs of MCQs was composed. The MCQs in each pair were similar concerning the same topic, but this similarity was not evident for an examinee that did not possess adequate knowledge on the topic addressed in the questions of the pair. The scoring of the paired questions avoided the procedure of mixed-scoring, i.e., both positive and negative markings, while at the same time a pair-wise bonus/penalty scoring rule was adopted. The results of the "paired" MCQs examination, when using the pair-wise scoring rule, were statistically indistinguishable with the grades produced by the OE, when made to the same sample of students, on the same topics and with the same levels of difficult. Both the results of the paired MCQs examination, when using the pair-wise scoring rule, and the OE results differed significantly from those obtained by scoring the same MCQs using a positive-grades-only scoring rule that ignored the pairing of MCQs.
2020
The use of computer-based examination systems offers advantages related to the reduction of human resource allocation and to gains in objectivity for the scoring process. Multiple-choice questions (MCQs) are widely used in such systems; one of the main reasons that hamper their effectiveness, in comparison with constructed response questions (CRQ), is the influence of guessing. Considering limitations within previously proposed MCQs examination methods and scoring rules, in the present work a novel MCQs examination method is presented, termed ‘adaptive’ MCQs method. MCQs are divided into 3 categories, per difficulty level. The ‘path’ that an examinee will follow is constituted by 3 phases, wherein a set of questions belonging to one of the three difficulty-categories, is appointed. The exact path followed is selected per the success level of the examinee in the preceding phase. The scoring provided by the adaptive MCQs examination method produced results that were statistically indi...
Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires
Concepts, Methodologies, Tools and Applications
The present chapter deals with the variants of grading schemes that are applied in current Multiple-Choice Questions (MCQs) tests. MCQs are ideally suited for electronic examinations, which, as assessment items, are typically developed in the framework of Learning Content Management Systems (LCMSs) and handled, in the cycle of educational and training activities, by Learning Management Systems (LMS). Special focus is placed in novel grading methodologies, that enable to surpass the limitations and drawbacks of the most commonly used grading schemes for MCQs in electronic examinations. The paired MCQs grading method, according to which a set of pairs of MCQs is composed, is presented. The MCQs in each pair are similar concerning the same topic, but this similarity is not evident for an examinee that does not possess adequate knowledge on the topic addressed in the questions of the pair. The adoption of the paired MCQs grading method might expand the use of electronic examinations, pr...
The use of multiple choice tests as a tool of assessment are popular in many university courses, particularly in the foundation year where there is large number of students taking common subjects such as mathematics or engineering science. Under such circumstances, the answers to the multiple choice questions (MCQs) are written on special forms with blank ovals. These forms are scanned and marking is done by comparing the answers with those inputs by the examiner. Subsequently, the results of the students are tabulated with the aid of customised software. However, this conventional method of administering MCQs is not practicable when the number of students and the number of questions are small e.g. ≤ 100 and ≤ 10, respectively. The present paper addresses this issue and discusses the development of an unconventional, yet an effective method for administering short and simple MCQs as a means of assessing student learning in a focus area of the subject matter. A MCQs test was designed...
Systems Based Assessment of Multiple Choice Questions (MCQS) for Quality Assurance in Testing
INTERNATIONAL JOURNAL OF EDUCATIONAL SCIENCES, 2020
This study aimed to assess the properties of MCQs using various courses in a novel holistic approach employing quantitative-comparative design. The data were collected from two university campuses in Saudi Arabia offering a Bachelor of Science in Nursing (BSN) program between 2018 and 2020. The multiple-choice questions on 'item difficulty' (34.36 ± 17.42), 'item discrimination' (0.32 ± 0.20), and 'option affinity' (0.32 ± 0.16) were reasonably good. The 'lexical density' was "Very Complex" (85.08 ± 13.37) with the Basic Adult Care Nursing final examination (91.54 ± 9.42). The 'readability index' was low (7.65 ± 3.08) in the Fundamentals of Nursing I Theory final examination (6.18 ± 3.13), but "High" in information entropy (4.23 ± 0.10) with the Fundamentals of Nursing I Theory examination (4.25 ± 0.11). Statistical differences (<.001) were noted in option affinity,' 'lexical density,' and 'readability index. Further, the multilayer perceptron (MLP) prediction model shows 'lexical density' (100%) as the most important. Findings indicate the need for quality assurance measures in the form of faculty training.
Multiple-Choice Tests - a Tool in the Assessing Knowledge
2010
The purpose of this paper is to analyse if Multiple-Choice Tests may be considered an interesting alternative for assessing knowledge, particularly in the Mathematics area, as opposed to the traditional methods, such as open questions exams. In this sense we illustrate some opinions of the researchers in this area. Often the perception of the people about the construction of this kind of exams is that they are easy to create. But it is not true! Construct well written tests it’s a hard work and needs writing ability from the teachers. Our proposal is analyse the construction difficulties of multiple choice tests as well some advantages and limitations of this type of tests. We also show the frequent critics and worries, since the beginning of this objective format usage. Finally in this context some examples of Multiple-Choice Items in the Mathematics area are given, and we illustrate as how we can take advantage and improve this kind of tests.
E- Assessment with Multiple-Choice Questions: A 5 Year Study of Students’ Opinions and Experience
Journal of Information Technology Education: Innovations in Practice
Aim/Purpose: The aim of this study is to understand student’s opinions and perceptions about e-assessment when the assessment process was changed from the traditional computer assisted method to a multiple-choice Moodle based method. Background: In order to implement continuous assessment to a large number of students, several shifts are necessary, which implies as many different tests as the number of shifts required. Consequently, it is difficult to ensure homogeneity through the different tests and a huge amount of grading time is needed. These problems related to the traditional assessment based on computer assisted tests, lead to a re-design of the assessment resulting in the use of multiple-choice Moodle tests. Methodology: A longitudinal, concurrent, mixed method study was implemented over a five-year period. A survey was developed and carried out by 815 undergraduate students who experienced the electronic multiple-choice questions (eMCQ) assessment in the courses of the IS ...