The Scoring of Matching Questions Tests: A Closer Look (original) (raw)
Related papers
The use of multiple choice tests as a tool of assessment are popular in many university courses, particularly in the foundation year where there is large number of students taking common subjects such as mathematics or engineering science. Under such circumstances, the answers to the multiple choice questions (MCQs) are written on special forms with blank ovals. These forms are scanned and marking is done by comparing the answers with those inputs by the examiner. Subsequently, the results of the students are tabulated with the aid of customised software. However, this conventional method of administering MCQs is not practicable when the number of students and the number of questions are small e.g. ≤ 100 and ≤ 10, respectively. The present paper addresses this issue and discusses the development of an unconventional, yet an effective method for administering short and simple MCQs as a means of assessing student learning in a focus area of the subject matter. A MCQs test was designed...
2020
The use of computer-based examination systems offers advantages related to the reduction of human resource allocation and to gains in objectivity for the scoring process. Multiple-choice questions (MCQs) are widely used in such systems; one of the main reasons that hamper their effectiveness, in comparison with constructed response questions (CRQ), is the influence of guessing. Considering limitations within previously proposed MCQs examination methods and scoring rules, in the present work a novel MCQs examination method is presented, termed ‘adaptive’ MCQs method. MCQs are divided into 3 categories, per difficulty level. The ‘path’ that an examinee will follow is constituted by 3 phases, wherein a set of questions belonging to one of the three difficulty-categories, is appointed. The exact path followed is selected per the success level of the examinee in the preceding phase. The scoring provided by the adaptive MCQs examination method produced results that were statistically indi...
Computers & Education, 2010
The aim of the present research was to compare the use of multiple-choice questions (MCQs) as an examination method, to the examination based on constructed-response questions (CRQs). Despite that MCQs have an advantage concerning objectivity in the grading process and speed in production of results, they also introduce an error in the final formulation of the score. The error is traced to the probability of answering a question by chance or based on an instinctive feeling, which does not enable the ascertainment of the knowledge of the whole background included in the question. In the present study, both MCQ and CRQ tests were given to examinees, in the framework of a computer-based learning system. Avoiding the procedure of mixed scoring, e.g. both positive and negative markings, a set of pairs of MCQs was composed. The MCQs in each pair were similar concerning the same topic, but this similarity was not evident for an examinee that did not possess adequate knowledge on the topic addressed in the questions of the pair. The examination based on these ''paired" MCQs, by using a suitable scoring rule, when made to the same sample of students, on the same topics and with the same levels of difficulty, gave results that were statistically indistinguishable with the grades produced by an examination based on CRQs, while both the ''paired" MCQ test results and the CRQ test results differed significantly from those obtained from a MCQ test using positive-only scoring rule.
2016
ABSTRACT. The aim ofsthis study was to show that the superiority of corrected-for-guessing scores over nuiber right scores, as true score estimates depeAds on the ability of examinees to recognize situations where they can eliminate one or more alternatives as-incorrect andto omit items.where they would only be guessing randomly. Previous ' investigations'show examinees lack this ability. An instructional strategy for teaching examinees to use pertial information in taking multiple choide tests under formula scoring conditions was tested on 280 fourth, fifth and sixth grade, private and publiC school students from the United-States Virgin Islands. 'Ma * method is outlined and followed by sections dealing with the administration of pre-tests and post- tests ' and the provisiOn of training to selected groups (in the form of an algorithm ffm attacking test Items, and practice sessions with immediate feedback). It 1 8 PAicated it the final.discussion of.research resu...
A computer-aided environment for construction of multiple-choice tests
2005
Multiple-choice tests have proved to be an efficient tool for measuring students' achievement and are used on a daily basis both for assessment and diagnostics worldwide. Statistics suggest that the Question Mark Computing Ltd.'s testing software Perception alone has had more ...
The objective of this study is to determine the item difficulties of tests that are going to be used on a Computer Adaptive Testing (CAT). According to the student responses, item difficulties have been found using different approaches. Consequently, best approach to find item difficulty has been determined by a simple classification tool. By using this classification tool, the best method to find item difficulties is determined and items have been classified using Rapidly-exploring Random Tree (RRT) algorithm. This classification ended up with two different results that define the future work of this study. First one tries to reveal which algorithm is the best method for classifying the questions for CAT software and the second one has determined whether the size of item pool is important for question classification. INTRODUCTION In education, computers can be used to deliver the course content and to measure the achievement in courses. This thesis dwells upon the assessment techniques in education via computers. The British Standards Institution estimates that approximately 1000 computer-assisted assessments are performed each day in the United Kingdom [1]. Computer delivered assessments focuses on two specific delivery methods: computer-based test (CBT) and computer-adaptive test (CAT). It have been reported numerous benefits of the CBT approach over the standard paper-and-pencil one [2-4]. These benefits ranged from the automation of marking and subsequent reduction in marking workload to the opportunity to provide students with immediate feedback on their performance [5]. Due to the limitations of traditional success measuring rates, the contribution for the understanding of students' knowledge about a subject is little. The use of CBT is an attempt to overcome these limitations and go some way to make course and assessments more interesting and useful processes for both teachers and students. Notwithstanding these benefits, previous works by researchers suggested that CBTs have being viewed as unsatisfactory in terms of efficiency. The reason for this inefficiency is that the questions administered during a given CBT session are not tailored for the specific ability of an individual student. In a typical CBT, the same predefined set of questions is presented to all students participating in the assessment session, regardless of their ability [5]. The questions within this fixed set are typically selected in such a way that a broad range of ability levels, from low to advanced, is catered for [4]. In this scenario, it is accepted that high-performance students are presented with one or more questions that are below their level of ability. Similarly, low-performance students are presented with questions that are above their level of ability [4]. The underlying idea of a CAT is to offer each student a set of questions that is appropriate to their level of ability [4]. Generally, a CAT initiates with a random question with an average difficulty. A more difficult question follows if the student answers the question correctly. Conversely, if the response provided is incorrect, an easier question that is suitable for this new lower estimate is presented next [6-7]. The use of CAT has been increasing and indeed replacing traditional CBTs in some areas of education and training [5]. Usually this replacement is associated with the need for higher efficiency when assessing large numbers, for example, in online training. The study presented here focuses on the evaluation of determining item difficulties for CAT software.
Automated Assessment with Multiple-choice Questions using Weighted Answers
Proceedings of the 13th International Conference on Computer Supported Education, 2021
A resource that has been used increasingly in order to assess people is the evaluation through multiple-choice questions. However, in many cases some test alternatives are wrong just because of a detail and scoring nought for them can be counter-pedagogical. Because of that, we propose an adaptation of the open-source system MCTest, which considers weighted test alternatives. The automatic correction is carried out by a spreadsheet that stores the students' responses and compares them with the individual answer keys of the corresponding test issues. Applicable to exams either in hardcopy or online, this study was validated to a total of 607 students from three different courses: Networks & Communications, Nature of Information, and Compilers.