Validity in Computer-Based Testing: a Literature Review of Comparability Issues and Examinee Perspectives (original) (raw)
Related papers
Computer-Based Testing and Validity: A Look Back and Into the Future
Assessment in Education: Principles, Policy & Practice, 2003
Test developers and organisations that rely on test scores to make important decisions about students and schools are aggressively embracing computer-based testing. As one example, during the past two years, 16 US states have begun to develop computerbased tests that will be administered to students across their state. Without question computer-based testing promises to improve the efficiency of testing and reduce the costs associated with printing and delivering paper-based tests. Computer-based testing may also assist in providing accommodations to students with special needs. However, differences in prior computer experience and the degree to which items from different content areas can be presented and performed on computer ranges widely. In turn, these factors will have different impacts on the validity of test scores. In this paper, we examine the potential benefits and costs associated with moving current paper-based tests to computer, with a specific eye on how validity might be impacted.
Computer-based Testing and Validity: a look back into the future
Assessment in Education: Principles, Policy & Practice, 2003
Test developers and organisations that rely on test scores to make important decisions about students and schools are aggressively embracing computer-based testing. As one example, during the past two years, 16 US states have begun to develop computerbased tests that will be administered to students across their state. Without question computer-based testing promises to improve the efficiency of testing and reduce the costs associated with printing and delivering paper-based tests. Computer-based testing may also assist in providing accommodations to students with special needs. However, differences in prior computer experience and the degree to which items from different content areas can be presented and performed on computer ranges widely. In turn, these factors will have different impacts on the validity of test scores. In this paper, we examine the potential benefits and costs associated with moving current paper-based tests to computer, with a specific eye on how validity might be impacted.
Computer-based tests: the impact of test design and problem of equivalency
Computers in Human Behavior, 2007
Nowadays, computerized forms of psychodiagnostic methods are often produced without providing appropriate psychometric characteristics, or without proving equivalency with conventional forms. Moreover, there exist tests with more than one computerized versions, which are mostly designed differently. Study I focused on the impact of test design. It was found that even simple change of color scheme (light stimuli on dark background vs. dark stimuli on light background) had a significant effect on subjects' performance. Study II examined equivalency of a computerized speeded test, which is broadly used within psychological practitioners in the Czech Republic; this form was found non-equivalent with its conventional counterpart.
Comparing Between Computer based Tests and Paper-and-Pencil based Tests
Testing subject has many subsets and connections. One important issue is how to assess or measure students or learners. What would be our tools, what would be our style, what would be our goal and so on. So in this paper the author attended to the style of testing in school and other educational settings. Since the purposes of educational system have been changed, there have been some inevitable changes in testing style. Therefore, in recent years, computer-based testing (CBT) has grown in popularity and will likely become the primary mode for delivering tests in the future. So different aspects of computer-based tests and paper-and-pencil based tests like motivation, anxiety and students' performance with different styles will be reviewed in this paper.
Assessing the Equivalence of Internet-Based vs. Paper-and-Pencil Psychometric Tests
The equivalence of paper-and-pencil (P&P) and computer-based tests (CBTs) has become an important focus of research in the past 20 years. However, few studies have specifically addressed the equivalence of Internet-based tests (IBTs) and P&P administrations of high-stakes admissions tests (Potosky & Bobko, 2004). Despite the fact that there is a shortage of evidence with regard to the equivalence of scores obtained in the IBT and P&P modalities, the number of tests administered via the Internet is constantly rising. The goal of the present study was to compare the scores of examinees who took the P&P version of a scholastic ability test with the scores of those who took it via the Internet. The study was conducted using the Psychometric Entrance Test used for admission to institutions of higher education in Israel. 370 examinees participated in the study. Half were given a Web-based format in a computer lab and the other half were given the same test in P&P format. The study confirmed the equivalence between IBT and traditional P&P versions of the test for the sample.
A Comparison of Computer Based Testing and Paper and Pencil Testing
2018
Today’s schools turn to computers for all aspects of learning, including assessment. While advantages to computer testing do exist, the comparability between paper pencil tests (PPT) and computer-based tests (CBT) must be considered. This study examined whether the testing medium impacts student performance in math assessment by addressing three questions. First, does a test mode effect exist, as evidenced by mean score difference between a CBT and a PPT? Second, does question type: multiple choice, constructed response, or extended response, relate to student performance? Third, does either gender or computer experience and familiarity impact CBT and PPT scores? Eighty 6th grade students took math tests with half of the questions on a PPT and half of the questions on a CBT. A computer familiarity survey was completed prior to the unit tests. Significant differences were found for one of the unit tests and for some of the question types.
International Journal of Emerging Technologies in Learning (iJET)
Score equivalency of two Computer-Based Testing (henceforth CBT) and Paper-and-Pencil-Based Testing (henceforth PBT) versions has turned into a controversial issue during the last decade in Iran. The comparability of mean scores obtained from two CBT and PBT formats of test should be investigated to see if test takers' testing performance is influenced by the effects of testing administration mode. This research was conducted to examine score equivalency across modes as well as the relationship of gender, and testing mode preference with test takers' performance on computerized testing. The information on testing mode preference and attitudes towards CBT and its features was supported by a focus group interview. Findings indicated that the scores of test takers were not different in both modes and there was no statistically significant relationship between moderator above variables and CBT performance.
Sustainability
Since schools cannot use face-to-face tests to evaluate students’ learning effectiveness during the COVID-19 pandemic, many schools implement computer-based tests (CBT) for this evaluation. From the perspective of Sustainable Development Goal 4, whether this type of test conversion affects students’ performance in answering questions is an issue worthy of attention. However, studies have not yielded consistent findings on the equivalence of the scores of examinees’ answering performance on computer-based tests (CBT) and paper-and-pencil tests (PPT) when taking the same multiple-choice tests. Some studies have revealed no significant differences, whereas others have exhibited significant differences between the two formats. This study adopted a counterbalanced experimental design to investigate the effects of test format, computerised presentation type, difficulty of item group, and administration order of item groups of different difficulty levels on examinees’ answering performance...
Developing standards for computerized psychological testing
Computers in Human Behavior, 1985
The growth of computerized psychological testing (CP7) require.~ that we analyze its ethical and legal ramifications. The response of the legal community and the profession of psychology is reviewed. There are potential problems with the administration of tests by computer, especially the interpretation of computer-administered tests with norms and validity data .from paper-and-pencil administrations, and the impact of computerization on certain test-taker groups and item !Tpes. Criteria for assessing the adequacy of clasr{~'calion systems used go a.~s~i~n peop# to interpretive statements, and the vahdity of computer-generated reports, are analyzed. CPT users should review r@orts and developers should disclose the rationale underlying interpretations. Not only the scientific merit, Dug also the fairness and efficiency of CPT will determine whether its potential isjully realized.
With promulgation of computer technology in educational testing, computerized testing (henceforth CBT) as green computing strategy is gaining popularity due to its advantages such as effective administration, flexible scheduling and immediate feedback over its conventional paper-based testing (henceforth PBT). Since some testing programs have begun to offer both versions of a test simultaneously, the effectiveness of CBT is queried by some scholars. Regarding to this aim, this study investigated the score equivalency of a test taken by 228 Iranian undergraduate students studying at a state university located in Chabahar region of Iran to see whether scores of two administrations of testing mode were equivalent. Then, two versions of the test were administered to the participants of two testing groups on four testing occasions in a counter balanced administration sequence with four weeks interval. One-Way ANOVA and Pearson Correlation tests were used to compare the mean scores and to find the relationship of testing order, computer attitudes and testing mode preference with testing performance. Findings of the study revealed that the scores of test takers were not different in both modes and the moderator variables were not considered external factors that might affect students' performance on CBT.