Use of multiple performance and symptom validity measures: Determining the optimal per test cutoff for determination of invalidity, analysis of skew, and inter-test correlations in valid and invalid performance groups (original) (raw)
Related papers
Applied Neuropsychology: Adult, 2019
The Test of Memory Malingering (TOMM) and Word Memory Test (WMT) are among the most well-known performance validity tests (PVTs) and regarded as gold standard measures. Due to the many factors that impact PVT selection, it is imperative that clinicians make informed clinical decisions with respect to additional or alternative PVTs that demonstrate similar classification accuracy as these well-validated measures. The present archival study evaluated the agreement/classification accuracy of a large battery consisting of multiple other freestanding/embedded PVTs in a mixed clinical sample of 126 veterans. We examined failure rates for all standalone/embedded PVTs using established cut-scores and calculated pass/fail agreement rates and diagnostic odds ratios for various combinations of PVTs using the TOMM and WMT as criterion measures. TOMM and WMT demonstrated the best agreement, followed by Word Choice Test (WCT). The Rey Fifteen Item Test had an excessive number of false-negative errors and reduced classification accuracy. The Digit Span age-corrected scaled score (DS-ACSS) had highest agreement. Findings lend further support to the use of a combination of embedded and standalone PVTs in identifying suboptimal performance. Results provide data to enhance clinical decision making for neuropsychologists who implement combinations of PVTs in a larger clinical battery.
Performance validity assessment using response time on the Warrington Recognition Memory Test
The Clinical Neuropsychologist, 2020
Objective: The present study tested the incremental utility of response time (RT) on the Warrington Recognition Memory Test-Words (RMT-W) in classifying bona fide versus feigned TBI. Method: Participants were 173 adults: 55 with moderate to severe TBI, 69 healthy comparisons (HC) instructed to perform their best, and 49 healthy adults coached to simulate TBI (SIM). Participants completed a computerized version of the RMT-W in the context of a comprehensive neuropsychological battery. Groups were compared on RT indices including mean RT (overall, correct trials, incorrect trials) and variability, as well as the traditional RMT-W accuracy score. Results: Several RT indices differed significantly across groups, although RMT-W accuracy predicted group membership more strongly than any individual RT index. SIM showed longer average RT than both TBI and HC. RT variability and RT for incorrect trials distinguished SIM-HC but not SIM-TBI comparisons. In general, results for SIM-TBI comparisons were weaker than SIM-HC results. For SIM-HC comparisons, classification accuracy was excellent for all multivariable models incorporating RMT-W accuracy with one of the RT indices. For SIM-TBI comparisons, classification accuracies for multivariable models ranged from acceptable to excellent discriminability. In addition to mean RT and RT on correct trials, the ratio of RT on correct items to incorrect items showed incremental predictive value to accuracy. Conclusion: Findings support the growing body of research supporting the value of combining RT with PVTs in discriminating between verified and feigned TBI. The diagnostic accuracy of the RMT-W can be improved by incorporating RT.
Psychological Injury and Law, 2018
Assessment of performance validity is an essential part of a neuropsychological evaluation, with the inclusion of two or more performance validity tests (PVTs) becoming routine practice. Considering the time to administer multiple tests, there has been some support for use of the Test of Memory Malingering (TOMM) Trial 1 (T1) as an independent, Bone and done^PVT. Notably, cutoffs for TOMM T1 need further validation, with an emphasis on minimizing false-positive classifications among those with bona fide cognitive impairment. In a clinically referred sample of 127 veterans, this study examined the role of cognitive impairment in TOMM performance and the utility of a TOMM T1 as an independent PVT. Examinees were administered the TOMM and three additional PVTs as part of a comprehensive neuropsychological battery. Sixty-eight percent of examinees were classified valid (35% of valid examinees were cognitively impaired). TOMM T1 ≤ 40 had excellent observed sensitivity (83%) and specificity (93%) overall, with minimal false-positive classification. TOMM T1 was also significantly correlated and concordant with other memory-based PVTs. Given score ranges and failure rates for TOMM T1 ≤ 40 among those with neurological/neurocognitive conditions, scores in the 37-40 range may merit administration of additional TOMM trials to maximize accuracy in identifying valid-cognitively impaired versus noncredible performance. Otherwise, an abbreviated TOMM administration (i.e., only T1) using a cutoff of ≤ 40-in conjunction with one or more additional PVTs-may be sufficient for detecting noncredible/invalid test performance in the absence of known or suspected neurological/ neurocognitive disorders.
Embedded Symptom Validity Tests and Overall Neuropsychological Test Performance
Archives of Clinical Neuropsychology, 2011
A sample of 314 consecutive clinical and forensic referrals with mild traumatic brain injury was evaluated using the Meyers Neuropsychological Battery (MNB). A comparison was made of the test performance and performance on the embedded Symptom Validity Tests (SVTs) with a control for multicolinearity utilized. Using the nine embedded SVTs in the MNB, the incidence of poor effort fell at 26% of the total sample. Involvement in litigation was related to more failures on the individual SVTs. The correlation between failed effort measures and the Overall Test Battery Mean (OTBM) was consistently negative, regardless of litigation status, in that more failures were associated with lower OTBM scores. The correlation between the number of SVTs failed and the OTBM was 2.77. Our results are similar to those presented by Green, Rohling, Lees-Haley, and Allen (2001); who reported a .73 correlation with the failure on the Word Memory Test and performance on the OTBM. The results of the current study also indicate that 50% of the variance in neuropsychological testing can be accounted by failures on internal SVTs.
Applied Neuropsychology: Adult, 2019
Use of multiple performance validity tests (PVTs) may best identify invalid performance, though few studies have examined the utility and accuracy of combining PVTs. This study examined the following PVTs in the Advanced Clinical Solutions (ACS) package to determine their utility alone and in concert: Word Choice Test (WCT), Reliable Digit Span (RDS), and Logical Memory Recognition (LMR). Ninety-three veterans participated in clinical neuropsychological evaluations to determine presence of cognitive impairment; 25% of the performances were deemed invalid via criterion PVTs. Classification accuracy of the ACS measures was assessed via receiver operating characteristic curves, while logistic regressions determined utility of combining these PVTs. The WCT demonstrated superior classification accuracy compared to the two embedded measures of the ACS, even in veterans with cognitive impairment. The two embedded measures (even when used in concert) exhibited inadequate classification accuracy. A combined model with all three ACS PVTs similarly demonstrated little benefit of the embedded indicators over the WCT alone. Results suggest the ACS WCT has utility for detecting invalid performance in a clinical sample with likely cognitive impairment, though the embedded ACS measures (RDS and LMR) may have limited incremental utility, particularly in individuals with cognitive impairment.
The Clinical Neuropsychologist, 2019
Objective: While the Neuropsychological Assessment Battery, Screening Module (S-NAB) is a commonly used cognitive screening measure, no composite embedded performance validity test (PVT) formula has yet been described within it. This study sought to empirically derive PVT formulas within the S-NAB using an analog simulation paradigm. Method: Seventy-two university students (M age ¼ 18.92) were randomly assigned to either an Asymptomatic (AS) or simulated mild traumatic brain injury (S-mTBI) group and were administered a neuropsychological test battery that included the S-NAB and standalone and embedded PVTs. The AS group was instructed to perform optimally, and the S-mTBI group received symptom and test coaching to help simulate mTBI-related impairment. Both groups received warnings regarding the presence of PVTs throughout the test battery. Results: Groups showed significant differences (all ps < .001) on all S-NAB domain scores and PVTs. In the S-NAB, the Attention (S-ATT) and Executive Function (S-EXE) domains showed the largest effect sizes (Cohen's ds ¼ 2.02 and 1.79, respectively). Seven raw scores from S-ATT and S-EXE subtests were entered as predictor variables in a direct logistic regression (LR). The model accurately classified 90.3% of cases. Two PVT formulas were described: (1) an exponentiated equation from LR results and (2) an arithmetic formula using four individually meaningful variables. Both formulas demonstrated outstanding discriminability between groups (AUCs ¼ .96-.97) and yielded good classification statistics compared to other PVTs. Conclusions: This study is the first to describe composite, embedded PVT formulas within the S-NAB. Implications, limitations, and appropriate future directions of inquiry are discussed.
Psychological Injury and Law, 2020
While literature on performance validity tests (PVTs) in neuropsychological assessment has examined memory-based paradigms, other research has suggested that tests of attention, visuospatial ability, and language may also detect noncredible performance. Previous work has identified several PVTs in the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS), though all of them emphasize memory-based subtests. This study sought to determine if PVT formulas can be derived from exclusively non-memory RBANS subtests (i.e., Figure Copy, Line Orientation, Picture Naming, Semantic Fluency, Digit Span, and Coding) using an analog simulation study. Seventy-two undergraduate participants (M age = 18.9) were assigned to either an asymptomatic (AS) group, which was instructed to perform optimally, or a simulated mild traumatic brain injury (S-mTBI) group, which received symptom and test coaching to help simulate mTBI-related impairment. Participants were administered a battery of neuropsychological tests, including the RBANS and standalone PVTs. Differences were found between groups for all RBANS subtests of interest except Picture Naming. Five subtests showing meaningful group differences were entered as predictor variables as one set in logistic regressions (LR); raw and norm-based scores were considered separately. Both LRs accurately classified 90.3% of cases with good sensitivity (.89) while maintaining ideal specificity (.92). Two exponentiated equations were described from LR results, with both yielding good discriminability (AUCs = .94), generally comparable with other PVTs. These findings suggested that non-memory RBANS subtests may be sensitive to noncredible performance and reiterate the importance of considering tests of various cognitive abilities when assessing performance validity during neuropsychological assessment. Limitations of this study and directions for future inquiry, including necessity for validation in a clinical sample, were discussed.
Assessment, 2017
This study examined the Word Choice Test’s (WCT) utility as a performance validity test in a mixed clinical sample of veterans referred for neuropsychological evaluation. Participants completed Green’s Word Memory Test (WMT), WCT, and Test of Memory Malingering (TOMM) Trial 1. Using the WMT as the criterion for valid performance, logistic regressions examined the WCT and TOMM’s classification accuracy for those with and without cognitive impairment (CI). Receiver operating characteristic curves were used to establish cut scores which maximized the sensitivity/specificity of each measure. In those without CI, both tests showed good classification accuracy (86.7% and 85.0%, respectively). Among those with CI, the TOMM retained good classification accuracy (82.3%), while the WCT’s decreased considerably (69.4%). Optimal WCT cut scores differed based on impairment status, with a higher sensitivity/specificity trade-off among those with CI. Successful performance on the WCT appeared to r...
Evaluation of the WMS-III Rarely Missed Index in a Naïve Clinical Sample
Clinical Neuropsychologist, 2010
We examined the WMS-III Rarely Missed Index as a reliable predictor of fabrication of memory difficulties. A total of 31 outpatients referred for neuropsychological evaluation completed the WMS-III Logical Memory Delayed Recognition Test (LMDR) before having heard the stories and again after hearing the stories. Of the 30 items from the LMDR completed by participants who had not heard the stories, 5 were found to significantly differ from chance; only 1 of those items was found to do so in the original RMI studies. Conversely, of the six items in the original RMI study, only one was found to differ from chance in the present study. A Monte Carlo randomization of the original six RMI items found that 69% of random responders fell below cutoffs for incomplete effort, suggesting that those with memory impairment severe enough to result in random responding are likely to be classified as demonstrating less than optimal effort. An examination of response bias found no differences among negative, positive, or no-bias responders in terms of overall memory performance. However, RMI was indicative of incomplete effort more often with individuals presenting with a negative response bias. The present study provides evidence of the limitations of using the RMI to detect incomplete effort using the LMDR. The relative values of specificity versus sensitivity are particularly important in clinical evaluations, with the prevention of false accusations of malingering an important goal. The use of multiple indicators of effort is reinforced by this study, and the conservative interpretation of the RMI in particular is recommended.