Rasch Analysis Research Papers - Academia.edu (original) (raw)

The problem of dimensionality with respect to Raven’s Advanced Progressive Matrices (APM) specifically and, more generally, g or fluid intelligence, has been a long-standing issue. The present article reports two studies examining the... more

The problem of dimensionality with respect to Raven’s Advanced Progressive Matrices (APM) specifically and, more generally, g or fluid intelligence, has been a long-standing issue. The present article reports two studies examining the dimensionality of both the original Set II of the APM ( n = 506) and a short form ( n = 644), using principal component analysis and Rasch analysis. Although the results from the principal component analysis were equivocal, results from the Rasch analyses more strongly suggested that both forms of the test are best described as being multidimensional. Furthermore, comparison of items common to both forms indicated a context effect, thus making adaptive testing versions of this test difficult.

Working in a South African national context where investments have been made in the design, administration and collection of standardised assessment data in the form of the Annual National Assessments (ANAs), we ask: How can these... more

Working in a South African national context where investments have been made in the design, administration and collection of standardised assessment data in the form of the Annual National Assessments (ANAs), we ask: How can these investments be leveraged to focus attention on better learning, better teaching, and also better assessment? To stimulate debate relating to this question, we draw on an empirical base of Grade 3 ANA data from two public schools. We offer a descriptive account of how attainment in mathematics ANAs was analysed quantitatively and then qualitatively so that school managers and teachers in these schools could make decisions relating to improving their teaching. Further we present the results of a more sophisticated analytical technique (Rasch analysis) which was used to further make use of the ANA evidence to identify problematic aspects of the assessment process.

Previous research has not explored how romantic attachment, mindfulness and empathy uniquely contribute to relationship satisfaction. This study investigated predictors of relationship satisfaction with an Australian sample of 18-25 year... more

Previous research has not explored how romantic attachment, mindfulness and empathy uniquely contribute to relationship satisfaction. This study investigated predictors of relationship satisfaction with an Australian sample of 18-25 year olds (N = 211) using hierarchical multiple regression. The overall model accounted for a significant amount of variance, however, only gender, self-rated empathy and perceived partner empathy accounted for unique variance in relationship satisfaction. Factor analyses of the Revised Experience of Close Relationships (ECR-R) questionnaire showed spurious factor structure, and subsequent Rasch analyses indicated areas of refinement required for this measure. The shared variance between the predictors is suggested to indicate a common underlying variable of mentalizing capacity – the ability to attend to states of mind in oneself and others.

The rational number knowledge of student teachers, in particular the equivalence of fractions, decimals, and percentages, and their comparison and ordering, is the focus of this article. An instrument comprising multiple choice, short... more

The rational number knowledge of student teachers, in particular the equivalence of fractions, decimals, and percentages, and their comparison and ordering, is the focus of this article. An instrument comprising multiple choice, short answer and constructed response formats was designed to test conceptual and procedural understanding. Application of the Rasch model enables verification of whether the test content was consistent with the construct under investigation. The validation process was enabled by making explicit the expected responses according to the model versus actual responses by the students. The article shows where the Rasch model highlighted items that were consistent with the model and those that were not. Insights into both the construct and the instrument were gained. The test items showed good fit to the model; however, response dependency and high residual correlation within sets of items was detected. Strategies for resolving these issues are discussed in this article. We sought to answer the research question: to what extent does this test instrument provide valid information that can be used to inform teaching and learning of fractions? We were able to conclude that a refined instrument applied to first-year students at university provides useful information that can inform the teaching and learning of rational number concepts, a concept that runs through mathematics curricula from primary to university. Previously, most research on rational number concept has been conducted on young learners at school.

This study examines the psychometric properties of the Lee Fatigue and Energy Scales (visual analog version) using a Rasch model application. The relationship between fatigue and energy is also described for a convenience sample of 102... more

This study examines the psychometric properties of the Lee Fatigue and Energy Scales (visual analog version) using a Rasch model application. The relationship between fatigue and energy is also described for a convenience sample of 102 women with human immunodeficiency virus/acquired immunodeficiency syndrome (HIV/AIDS) who completed the Lee Fatigue and Energy Scales in the morning and evening. Both scales were assessed for internal scale validity, unidimensionality, and uniform differential item functioning in relation to morning and evening ratings. Analyses confirmed that both the Fatigue and Energy Scales demonstrated evidence of internal scale validity and unidimensionality. Mean fatigue measures were also higher in the evening than in the morning and mean energy measures were higher in the morning than in the evening (both p<0.001), indicating that time of day is an important consideration. Fatigue and energy measures were moderately correlated with each other in the morning but not in the evening. The concepts of energy and fatigue were inversely related, but not polar opposites in this sample. Fatigue and energy may therefore be distinct constructs that should not be used interchangeably, either in measurement or when interpreting outcomes for research or clinical purposes

The Interaction with Disabled Persons scale (IDP) has been used in research into baseline attitudes and to evaluate whether a shift in attitudes towards people with developmental disabilities has occurred following some form of... more

The Interaction with Disabled Persons scale (IDP) has been used in research into baseline attitudes and to evaluate whether a shift in attitudes towards people with developmental disabilities has occurred following some form of intervention. This research has been conducted on the assumption that the IDP measures attitudes as a multidimensional construct and has good internal consistency. Such assumptions about

University entrances examinations are conducted to ensure qualified applicant are placed into appropriate programs of their choices. The outcomes of the test have an important and significant value in taking appropriate decision on the... more

University entrances examinations are conducted to ensure qualified applicant are placed into appropriate programs of their choices. The outcomes of the test have an important and significant value in taking appropriate decision on the applicant's eligibility, the validity of that examination is paramount to achieving the set goal. The aim of this study is to provide empirical evidence of the construct validity of the newly developed Economics Test using traditional Classical Test Theory and Rasch Measurement Model. The developed Economics Test consists of 70 items after expert judgment and review was administered to 280 students, age 16-20 randomly selected from two public schools in Kano. The study employed a CTT and Rasch model to analyze the data using ITEMAN 4.3 and WINSTEPS 3.72.3 software. The softwares automatically generate the recommended estimate of the parameters to judge the quality of the test items. The results of CTT identified 17 problematic items using difficulty and discriminating index. The results of Rasch showed person statistics (Separation 2.40>2.00 and reliability 0.85>0.80) and item statistics (separation 3.73>3.0 and reliability 0.93>0.8) an excellent person and item reliability. The test measures unidimensional construct supported by the raw variance of 24.9% explained by measures. Investigation of the item person map revealed that the test covered a wide range of the examinees' ability. Overall, using Rasch 10 misfitting construct irrelevant items were identified for deletion. While CTT provides information that is limited to two parameters, the Rasch results provide very detailed information on the quality of the test items. Thus both models can be integrated to generate enough evidence of validity and reliability items in the development of a standardize test.

Background and aims: The Bergen Social Media Addiction Scale (BSMAS), a six-item self-report scale that is a brief and effective psychometric instrument for assessing at-risk social media addiction on the Internet. However, its... more

Background and aims: The Bergen Social Media Addiction Scale (BSMAS), a six-item self-report scale that is a brief and effective psychometric instrument for assessing at-risk social media addiction on the Internet. However, its psychometric properties in Persian have never been examined and no studies have applied Rasch analysis for the psychometric testing. This study aimed to verify the construct validity of the Persian BSMAS using confirmatory factor analysis (CFA) and Rasch models among 2,676 Iranian adolescents. Methods: In addition to construct validity, measurement invariance in CFA and differential item functioning (DIF) in Rasch analysis across gender were tested for in the Persian BSMAS. Results: Both CFA [comparative fit index (CFI) = 0.993; Tucker–Lewis index (TLI) = 0.989; root mean square error of approximation (RMSEA) = 0.057; standardized root mean square residual (SRMR) = 0.039] and Rasch (infit MnSq = 0.88–1.28; outfit MnSq = 0.86–1.22) confirmed the unidimensionality of the BSMAS. Moreover, measurement invariance was supported in multigroup CFA including metric invariance (ΔCFI = −0.001; ΔSRMR = 0.003; ΔRMSEA = −0.005) and scalar invariance (ΔCFI = −0.002; ΔSRMR = 0.005; ΔRMSEA = 0.001) across gender. No item displayed DIF (DIF contrast = −0.48 to 0.24) in Rasch across gender. Conclusions: Given the Persian BSMAS was unidimensional, it is concluded that the instrument can be used to assess how an adolescent is addicted to social media on the Internet. Moreover, users of the instrument may comfortably compare the sum scores of the BSMAS across gender.

A questionnaire Usability Self Learning Module is developed to measure the overall development of the learning module Adobe Photoshop constructs involving three modules, module’s usability, applying theory of the cognitive load and... more

A questionnaire Usability Self Learning Module is developed to measure the overall
development of the learning module Adobe Photoshop constructs involving three modules, module’s
usability, applying theory of the cognitive load and minimalist. The study was conducted to produce
empirical data on the reliability and validity of the Usability of Adobe Photoshop’s Module
questionnaire using the Rasch Measurement Model. A survey was conducted on 120 participants who
attended the course Adobe Photoshop using the modules studied. The questionnaire is distributed
using one of the Web 2.0 technology platform, Google Docs. The reliability and validity of the
Usability Adobe Photoshop’s module questionnaire were tested with the Rasch Measurement Model
using the Winstep version 3.69.1.11 program.The Rasch analysis showed that person reliability index
is 0.87 and item reliability of 0.94. In term of item polarity, each item was able to contribute to
measuring the effectiveness of the learning module development measurement since the PTMEA
CORR exceeded 0.30, in specific between 0.50 to 0.78. There are 27 items in the questionnaire, which
is divided into 3 different constructs. Each construct contains 7 to 11 items. The unidimensionality test
conducted showed standardized residuals variance of 49.8%, with the biggest secondary dimension
in the first constrast amounting variance of 6.5%. Item fit analysis showed that none of the items
needed to be dropped since infit mean square values are between 0.73 and 1.39, and the outfit mean
square values are between 0.70 and 1.40. Items in the questionnaires are ordered in a continuum of
increasing intensity for the measurement of the Usability of Module construct. This shows the validity
of the constructs in Usability of the module.The Rasch Measurement Model shows that the Usability
of the Module has a high reliability and validity to be used for measuring the usability of the module
and the application of cognitive load theory and the theory of minimalism.

The aim of the present study was to evaluate the psychometric properties of the Brazilian version of the Nottingham Health Profile (NHP), a generic instrument for quality of life assess- ment. This NHP was translated to Portuguese,... more

The aim of the present study was to evaluate the psychometric properties of the Brazilian version of the Nottingham Health Profile (NHP), a generic instrument for quality of life assess- ment. This NHP was translated to Portuguese, following recommended methodology, and ap- plied to 215 subjects, being 170 community el- derly (66.3 ± 8.5 years), 30 chronic stroke sur- vivors

The recently developed Fear of COVID-19 Scale (FCV-19S) is a seven-item uni-dimensional scale that assesses the severity of fears of COVID-19. Given the rapid increase of COVID-19 cases in Bangladesh, we aimed to translate and validate... more

The recently developed Fear of COVID-19 Scale (FCV-19S) is a seven-item uni-dimensional scale that assesses the severity of fears of COVID-19. Given the rapid increase of COVID-19 cases in Bangladesh, we aimed to translate and validate the FCV-19S in Bangla. The forward-backward translation method was used to translate the English version of the questionnaire into Bangla. The reliability and validity properties of the Bangla FCV-19S were rigorously psychometrically evaluated (utilizing both confirmatory factor analysis and Rasch analysis) in relation to socio-demographic variables, national lockdown variables, and response to the Bangla Health Patient Questionnaire. The sample comprised of 8550 Bangladeshi participants. The Cronbach α value for the Bangla FCV-19S was 0.871 indicating very good internal reliability. The results of the confirmatory factor analysis showed that the unidimensional factor structure of the FCV-19S fitted well with the data. The FCV-19S was significantly correlated with the nine-item Bangla Patient Health Questionnaire (PHQ-90) (r = 0.406, p < 0.001). FCV-19S scores were significantly associated with higher worries concerning lockdown. Measurement invariance of the FCV-19S showed no differences with respect to age or gender. The Bangla version of FCV-19S is a valid and reliable tool with robust psychometric properties which will be useful for researchers carrying out studies among the Bangla speaking population in assessing the psychological impact of fear from COVID-19 infection during this pandemic.

College students may view research methods courses with negative attitudes, however, few studies have investigated this issue due to the lack of instruments that measure the students' attitudes towards research. Therefore, the purpose... more

College students may view research methods courses with negative attitudes, however, few studies have investigated this issue due to the lack of instruments that measure the students' attitudes towards research. Therefore, the purpose of this study was to examine the psychometric properties of a Attitudes Toward Research Scale using Rasch rating scale analysis. Assessment of attitudes toward research is essential to determine if students have negative attitudes towards research and assist instructors in better facilitation of learning research methods in their courses. The results of this study have shown that a thirty item Attitudes Toward Research Scale yielded scores with high person and item reliability.

Quality measurement is essential in every form of research, including institutional research and assessment. This paper addresses the erroneous assumptions institutional researchers often make with regard to survey research and provides... more

Quality measurement is essential in every form of research, including institutional research and assessment. This paper addresses the erroneous assumptions institutional researchers often make with regard to survey research and provides an alternative method to producing more valid and reliable measures. Rasch measurement models are discussed and a demonstration is provided, thus highlighting the utility of the Rasch models in higher education research and practice.

Background This article describes the development and validation of a self-reported questionnaire, the KQoL-26, that is based on the views of patients with a suspected ligamentous or meniscal injury of the knee that assesses the impact of... more

Background This article describes the development and validation of a self-reported questionnaire, the KQoL-26, that is based on the views of patients with a suspected ligamentous or meniscal injury of the knee that assesses the impact of their knee problem on the quality of their lives. Methods Patient interviews and focus groups were used to derive questionnaire content. The instrument was assessed for data quality, reliability, validity, and responsiveness using data from a randomised trial and patient survey about general practitioners' use of Magnetic Resonance Imaging for patients with a suspected ligamentous or meniscal injury. Results Interview and focus group data produced a 40-item questionnaire designed for self-completion. 559 trial patients and 323 survey patients responded to the questionnaire. Following principal components analysis and Rasch analysis, 26 items were found to contribute to three scales of knee-related quality of life: physical functioning, activity limitations, and emotional functioning. Item-total correlations ranged from 0.60–0.82. Cronbach's alpha and test retest reliability estimates were 0.91–0.94 and 0.80–0.93 respectively. Hypothesised correlations with the Lysholm Knee Scale, EQ-5D, SF-36 and knee symptom questions were evidence for construct validity. The instrument produced highly significant change scores for 65 trial patients indicating that their knee was a little or somewhat better at six months. The new instrument had higher effect sizes (range 0.86–1.13) and responsiveness statistics (range 1.50–2.13) than the EQ-5D and SF-36. Conclusion The KQoL-26 has good evidence for internal reliability, test-retest reliability, validity and responsiveness, and is recommended for use in randomised trials and other evaluative studies of patients with a suspected ligamentous or meniscal injury.

This research is concerned with the challenges for Building Information Modelling (BIM) adoption from the perspective of clients/owners in the Malaysian construction industry. A survey was administered using 33-item questionnaire... more

This research is concerned with the challenges for Building Information Modelling (BIM) adoption from the perspective of clients/owners in the Malaysian construction industry. A survey was administered using 33-item questionnaire specifically developed for the purpose of examining the challenges for BIM adoption in Malaysian. Data were analysed using WINSTEPS version 3.69.1.16 software to perform Rasch analysis. The findings of this research showed that although there was an indication that organisations in Malaysia were becoming more aware and interested in adopting BIM, these organisations were being hindered by lack of references and direction for BIM in the construction industry. The main contribution of this paper is to provide a baseline to understand the reason for slow BIM adoption in Malaysia and makes recommendations for future studies on how to mitigate these challenges for BIM adoption

Unlike classical test theory (CTT), where estimates of reliability are assumed to apply to all members of a population, item response theory provides a theoretical framework under which reliability can vary by test score. However,... more

Unlike classical test theory (CTT), where estimates of reliability are assumed to apply to all members of a population, item response theory provides a theoretical framework under which reliability can vary by test score. However, different IRT models can result in very different interpretations of reliability, as models that account for item quality (slopes) and probability of a correct guess significantly alter estimates. This is illustrated by fitting a TOEIC Bridge practice test to 1 (Rasch) and 3 parameter logistic models and comparing results. Under the Bayesian Information Criterion (BIC) the 3-parameter model provided superior fit. The implications of this are discussed.

To support teachers in facilitating students’ moral reasoning development as specified within the Singapore Ministry of Education Character and Citizenship Education curriculum, the Moral Reasoning Questionnaire (MRQ) was developed and... more

To support teachers in facilitating students’ moral reasoning development as specified within the Singapore Ministry of
Education Character and Citizenship Education curriculum, the Moral Reasoning Questionnaire (MRQ) was developed
and underwent preliminary validation. Based upon expert reviews, cognitive interviews and a classical test theorybased factor analytic approach, the development and preliminary validation found evidence (i.e., content appropriateness,
response processes and internal structure) to support the validity and reliability of the MRQ. This study aims to extend
the validation by examining the purported MRQ items and scale at a deeper level on the Rasch Measurement Theory,
given that it is the only model that presents appropriate properties of interval measurement on a log-linear scale. The
Rasch analysis found anomalies including differential item functioning and disordered thresholds in the initial set of
items. Upon remediation and a second Rasch analysis, the MRQ responses were consistent with that expressed by
the Rasch model (i.e., an item with an endorsability higher than what a respondent would tend to endorse would
have a lower probability of being endorsed than an item exhibiting an endorsability below what that respondent
would tend to endorse) and hence, there was sufficient evidence to support measurement invariance, and that MRQ
scores could be concluded to characterise persons invariantly across a continuum.