Predicting performance on a situational judgement test: the role of communication skills, listening skills, and expertise (original) (raw)

2007

In this chapter, we give an overview of situational judgment tests (SJTs) as selection instruments. Their history, basic characteristics, and development are presented. The available research evidence regarding their reliability, construct-related validity, criterionrelated validity, incremental validity, subgroup differences, and test-taker perceptions is also reviewed. As a general conclusion, the increasing popularity of SJTs in personnel selection seems to be accredited to their potential to capture a variety of constructs and for different purposes. Additionally, SJTs are able to predict several job-related and/or academic criteria while at the same time offering prospects permitting to select for diversity.

Situational judgment tests: An overview of current research

Human Resource Management Review, 2009

Situational judgment tests (SJTs) are popular personnel selection tests. To aid researchers, the paper summarizes the current knowledge and where knowledge gaps exist. To guide practice, the paper provides evidence-based recommendations. The paper begins with a brief history of SJTs, presents likely reasons for the resurgence of SJT research and practice, and summarizes the theoretical basis of SJTs. Then, the distinction between personnel selection methods and constructs is reviewed as it is particularly important in understanding SJTs. SJT research relevant to reliability and validity is summarized as is research relevant to the implementation of SJTs. The paper concludes with recommendations for practice and an agenda for future research.

Situational Judgment Tests, Response Instructions, and Validity: A Meta-Analysis

Personnel Psychology, 2007

Situational judgment tests (SJTs) are personnel selection instruments that present job applicants with work-related situations and possible responses to the situations. There are typically 2 types of instructions: behavioral tendency and knowledge. Behavioral tendency instructions ask respondents to identify how they would likely behave in a given situation. Knowledge instructions ask respondents to evaluate the effectiveness of possible responses to a given situation. Results showed that response instructions influenced the constructs measured by the tests. Tests with knowledge instructions had higher correlations with cognitive ability. Tests with behavioral tendency instructions showed higher correlations with personality constructs. Results also showed that response instructions had little moderating effect on criterion-related validity. Supplemental analyses showed that the moderating effect of response instructions on construct validity was not due to systematic differences in item content. SJTs have incremental validity over cognitive ability, the Big 5, and over a composite of cognitive ability and the Big 5.

Situational Judgment Tests: A Review of Practice and Constructs Assessed

International Journal of Selection and Assessment, 2001

In this article, we seek to summarize current practice concerning situational judgment tests in personnel selection. We begin by describing the manner in which situational judgment tests are developed and examining the diverse ways in which situational items are presented and scored. We then offer speculation concerning constructs assessed by situational judgment tests as well as discuss the legal aspect of situational judgment measures. We also review meta-analytic evidence concerning the construct validity of situational judgment tests and offer several new meta-analytic findings. Situational judgment tests are shown to be typically correlated moderately with general mental ability. Their primary personality correlates are emotional stability, conscientiousness, and agreeableness. Situational test scores also tend to increase with increasing years of job experience. The article concludes with a list of areas that need addressed in future research.

Situational judgment tests, response instructionns, and validity: A meta-analysis

2007

Situational judgment tests: a review of recent research

Personnel Review, 2008

Purpose -The purpose of this paper is to give an empirically-based review of the strengths and weaknesses of situational judgment tests (SJTs). Design/methodology/approach -The features, history, and development of SJTs are discussed. Next, a computerized search (from 1990-2007) is conducted to retrieve empirical studies related to SJTs. The effectiveness of SJTs is discussed in terms of reliability, criterion-related validity, incremental validity, construct-related validity, utility, adverse impact, applicant perceptions, fakability, and susceptibility to practice and coaching effects. Findings -Strengths of SJTs are that they show criterion-related validity and incremental validity above cognitive ability and personality tests. SJTs have also less adverse impact towards minorities (especially if the cognitive loading of the SJT is low). Furthermore, applicant reactions towards SJTs are positive and SJTs enable to test large applicant groups at once (through the Internet). In terms of weaknesses, SJTs might be prone to faking, practice, and coaching effects. There is also debate about what constructs are measured by SJTs. Research limitations/implications -Five avenues for future research are discussed: construct-related validity of SJTs, utility of SJTs vis-à -vis other predictors, impact of SJT features on validity and adverse impact, examination of alternative stimulus and response formats, and cross-cultural transportability of SJTs. Practical implications -Practitioners receive evidence-based information about the features, development, and strengths and weaknesses of SJTs. Originality/value -Apart from the USA, SJTs have not made strong inroads in selection practice in Europe and other parts of the world. This evidence-based paper might highlight the value of SJTs.

The internal structure of situational judgement tests reflects candidate main effects: Not dimensions or situations

Journal of Occupational and Organizational Psychology, 2016

Despite their popularity and capacity to predict performance, there is no clear consensus on the internal measurement characteristics of situational judgment tests (SJTs). Contemporary propositions in the literature focus on treating SJTs as methods, as measures of dimensions, or as measures of situational responses. However, empirical evidence relating to the internal structure of SJT scores is lacking. Using generalizability theory, we decomposed multiple sources of variance for three different SJTs used with different samples of job candidates (N 1 = 2,320; N 2 = 989; N 3 = 7,934). Results consistently indicated that (a) the vast majority of reliable observed score variance reflected SJT-specific candidate main effects, analogous to a general judgment factor and that (b) the contribution of dimensions and situations to reliable SJT variance was, in relative terms, negligible. These findings do not align neatly with any of the proposals in the contemporary literature; however they do suggest an internal structure for SJTs. Practitioner Points  To help optimize reliable variance, overall-level aggregation should be used when scoring SJTs.  The majority of reliable variance in SJTs reflects a general performance factor, relative to variance pertaining to specific dimensions or situations.  SJT developmental feedback should be delivered in terms of general SJT performance rather than on performance relating to specific dimensions or situations.  Generalizability theory, although underutilised in multifaceted measurement, offers an approach to informing on the psychometric properties of SJTs that is well-suited to the complexities of SJT measurement designs.

Refining situational judgment test methods

2006

Abstract: Situational Judgment Tests (SJTs) assess knowledge, skills, values, and attitudes. They present scenarios, which are based on real events, to be judged, understood, scaled, and interpreted by the examinee. These instruments have been used to evaluate cognitive theories and training programs, and to predict performance. This paper presents the argument that SJTs have potential to renew and reinvigorate many aspects of psychological measurement.

Best Practice Recommendations for Situational Judgment Tests

Australasian journal of organisational psychology, 2015

Situational judgement tests (SJTs) are increasingly popular as part of selection procedures because of their strong psychometric properties. SJTs present test takers with job-related situations and a series of potential responses. The aim of this review article is to provide evidence-based recommendations for practitioners who are considering implementing SJTs for their clients, as well as offering an analysis of some key issues for academics. This research provides a review of the literature on SJTs to answer three questions: (1) What is the difference between framing questions either as what should you do versus what would you do? (2) What is the difference between multimedia-based and text-based SJTs? (3) What is the best process for SJT creation? Following this, recommendations are provided as follows: That responses are optimally framed in terms of what a respondent thinks s/he should do, not what s/he would do; that multimedia-based SJTs are used in preference to text-based SJTs; and a 'best practice' process for creating an SJT is outlined. Finally, the review includes a discussion of implications for creating tests that are inclusive of different ethnic groups, as well as ethical issues relating to the use of SJTs in selection.

Situational Judgment Tests: An Overview of Development Practices and Psychometric Characteristics

Personnel Assessment and Decisions, 2020

Situational judgment tests (SJTs) are popular assessment methods often used for personnel selection and promotion. SJTs present problem scenarios to examinees, who then evaluate each response option for addressing the issue described in the scenario. As guidance for practitioners and researchers alike, this paper provides experience- and evidence-based best practices for developing SJTs: writing scenarios and response options, creating response instructions, and selecting a response format. This review describes scoring options, including key stretching and within-person standardization. The authors also describe research on psychometric issues that affect SJTs, including reliability, validity, group differences, presentation modes, faking, and coaching.

Predicting performance on a situational judgement test: the role of communication skills, listening skills, and expertise (original) (raw)

Related papers