Rainer Kurz - Academia.edu (original) (raw)

Books by Rainer Kurz

Kogan Page Book , 2008

Book Chapter

A book chapter outlining the origin of the Great 8 Competencies and the 'World of Work' (WoW) model.

Conference Presentations by Rainer Kurz

International Congress of Psychology (ICP) in Prague, 2024

This symposium brings together psychometric testing experts to share pioneering insights into adv... more This symposium brings together psychometric testing experts to share pioneering insights into advances in Computer Based Assessment (CBA).

The first paper reviews milestones in the development and evaluation of CBA systems based on Bartram & Bayliss (1984). Validation principles are illustrated through the correlation of personality, ability, interest, and motivation scales with Great 8 competency factors.

The second paper outlines contemporary advances in personality, ability and competency assessment and reporting through Computer-Based Test Interpretation (CBTI) in the light of Bartram (1994) which shaped BPS and EFPA test review processes. Versatile use of assessment data for multi-level reporting across tools are illustrated and backed with validation data.

The third paper illustrates applications of Sociomapping (Bahbouh, 2012) to psychometric assessment reporting at group and construct set level. Topographical representation of relationships between Big 5 and Great 8 constructs align to Stability and Plasticity in Cybernetic Big 5 Theory (DeYoung, 2015). Case studies on a 'Talent' group and a vet team are presented.

The fourth paper outlines a leading-edge approach involving continuous assessment of wellbeing which presents a measurement model for Employee Experience (EX) based on continuous adaptive micro interactions. The system captures trends in EX data in teams and organisations over time, while accounting for the dynamic relationship between contextual factors in the work environment and internal mindset factors that together influence an individual's EX.

The fifth paper outlines how test review processes have been impacted by advances in CBA. Issues include the use of item banks and CBTI reports products where reviewers will depend on technical documentation featuring psychometric data, especially validation results, to assess the appropriateness of the interpretations.

As artificial intelligence becomes more prevalent, it is timely to discuss past, present and future of CBA. The discussant slot will review the contributions and draw out themes, challenges, and opportunities.

Discussant: Dragos Iliescu, University of Bucharest

Intelligent Testing Systems: Past, Present and Future
Rainer Hermann Kurz, PhD
HUCAMA Analytics, London

Bartram & Bayliss (1984) wrote about the future use of computers in assessment. This paper reviews issues surrounding seven components of a fully automated Intelligent Testing System (ITS) and demonstrates how empirical data can be used to build valid system. 1. Test choice requires a sophisticated understanding of the assessment purpose, tools available and the importance of the scales. 2. Administration requires robustness against Intellectual Property content theft and faking. 3. Scoring requires transparency, sensitivity, and validity. 4.Interpretation needs to be based on valid information presented in understandable form 5. Feedback must be framed appropriate for the recipient whether candidates or 4rd parties. 6. Decision-making must be embedded within legal frameworks and proportionate to the quality of the evidence gathered. 7. Monitoring of outcomes is crucial to establish and enhance validity.

With the advent of AI there is a risk that poor assessment practices will proliferate especially if poor off-line processes are perpetuated and magnified through AI. On the other hand, AI that builds on and integrates sound testing practices could add value.

A data set where 250 individuals completed ability, personality, motivation, and competency assessments is utilised to illustrate data-driven development approaches. Predictors for the Great 8 Competencies (Kurz & Bartram, 2002) are calculated based on the meta-analysis of Bartram (2005) to illustrate how the scales of these five assessment modalities inter-relate. The two highest correlations for the factors are identified for each scale to facility the development of expert systems based on Great 8 and Big 5 models – whether using conventional or AI approaches.

Computer-Based Assessment across Personality, Ability and Competency Factors
Michele Guarini
HUCAMA Group, Copenhagen

This paper builds on the pioneering work of Bartram (1994) on Computer-Based Test Interpretation (CBTI) with a lens on group reporting based on the Great 8 Success Factors inspired by Kurz & Bartram (2002).

Personality Factors:

The modular range features general, professional, and executive level versions with 80, 160 and 240 questions measuring 16, 32 and 48 facets respectively grouped into 8 factors.

A. The Role Wheel Report uses ipsatised data (Bartram, 1996) to remove the effect of individual response style for enhanced group reporting.
B. The Leadership Report maps 48 personality facets to 8 Primary Colours of Leadership constructs (Pendelton, Furnham & Cowell, 2021) with an observed validity with external reviewer ratings of .50 (N=113).
C. The Aspects Reports covers emotional and operational themes that underpin potential and performance fully integrating Emotional Intelligence and Learning Agility constructs. The median construct convergence for the nearest counterpart scale was .63 for the 15 EQi 2.0 facets (N=101) and .70 for its five higher-order compounds.

Ability Factors:

This assessment consists of diagrammatic, numerical, spatial, and verbal component tests with a time limit of 8 minutes each. Rule-based item generation builds on Kurz (1990) and reporting on Kurz (2000) featuring Supra-scores across areas and sub-scores for speed, accuracy and caution.

Competency Factors:

This inventory features an overarching GETTING IT RIGHT, GETTING ALONG, GETTING AHEAD and GETTING ALIGNED model that builds on the Schwartz (1993) values circumplex and the Hogan & Holland (2004) view on performance. Each quadrant pairs up two Great 8 factors. Extreme tie-breaker data is collected and item level results shown.
A Potential & Performance Solution gap analysis tool brings together reporting across personality, ability and competency assessments for individuals and groups using Sten scores on a dynamic dashboard that enables multi-level integration, interpretation, and interrogation of data.

Sociomapping and Team Profile Analyzer in Psychometric Assessment
Pauline Willis
Lauriate, Australia

Sociomapping (Bahbouh, 2012) is an innovative method for tracking quality and frequency of communication in organisation. This paper outlines two applications of the underlying methodology to psychometric assessment.

STORM software uses scale correlations to produce a topographical ‘heat map’ that indicates the centrality of scales and maps out the relationship between constructs. Correlations (N=308) of Big 5 scales (based on NEO) with Great 8 (Kurz & Bartram, 2002) constructs revealed the centrality of Emotional Stability together with motivational (Need for Achievement & Power) constructs to the variable set. Stability (Alpha) vs Plasticity (Beta) meta-factors delineated one axis whereas People vs. Task delineated the other. The graph illustrates how constructs interrelate and facilitate understanding of the nature of ‘derailment’ scales. A cross-validation on N=466 largely confirm the results using different questionnaires.
Team Profile Analyzer (TPA) software produces a map that indicates the centrality of individuals to groups and similarities between group members. In the ‘Sociomap of Profile Similarity’ each group member is represented by a point and mutual distances represent mutual similarity of individual profiles. A heat map colour scheme indicates centrality. Personality assessment results for 16 ‘Elite’ performers across business, arts and sports were analysed using TPA. 10 group members shared many characteristics whereas 6 were different at the highest level of analysis with lower-level scores illuminating the origin of higher-order trends. The analysis explored differences between business leaders and others as well as sex differences. TPA analysis on Personality Factors results for a veterinary team will also be presented.

The applications demonstrate the power of group level reporting across psychometric results. The question arises how advances in academic theory building, such as the Periodic Table of Personality (Woods & Anderson, 2016) and Cybernetic Big 5 Theory (DeYoung, 2015), can be built upon and integrated with the Sociomapping methodology.

Insights in Motion: A Comprehensive Model for Tracking Employee Experience Over Time
Richard T. Justenhoven, PhD
Welliba, Germany

The varying stability across time and situations different constructs exhibit is well known and subject of ongoing research (Steyer et al., 2015). Talent assessment and management tools continue to evolve, and technological advancements enable capturing data in ever increasing breadth and depth.
This enables increasingly nuanced approaches to measuring constructs over time. This paper presents a measurement model for Employee Experience (EX) based on continuous adaptive micro interactions (CadaMint) as one example of this (Preuss et al., 2023). Grounded in Self-Determination-Theory (Deci & Ryan, 2000; Yeager & Dweck, 2020) and the Job-Demands-Resources Model (Bakker & Demerouti, 2007) CadaMint captures trends in EX data in teams and organisations over time, while accounting for the dynamic relationship between contextual factors in the work environment and internal mindset factors that together influence an individual's EX.

As data on EX often covers quarterly or annual cycles, CadaMint accounts for decreasing reliability of data on individual level as measurements age and increases when new measurements for the same constructs are added. This is achieved through a set of characteristics assigned to each instrument and variable, determining how measurements age. Interactions of different factors and fluctuations over time not only pose challenges to measurement, but also to the way outputs are presented to HR profes...

International Congress of Psychology, 2024

This paper outlines a synthesis of Cybernetic Big 5 Theory (DeYoung, 2015) and Great 8 Competenci... more This paper outlines a synthesis of Cybernetic Big 5 Theory (DeYoung, 2015) and Great 8 Competencies (Kurz & Bartram, 2002) to address vexing problems in psychometric assessment practice.

Digman (1997) found Alpha (Agreeableness, Conscientiousness & Emotional Stability) and Beta (Extraversion & Openness) higher-order meta-factors of the Big 5 that DeYoung, Peterson & Higgins (2001) renamed ‘Stability’ and ‘Plasticity’. Judge, Piccolo, & Ilies (2004) found high validity in the Ohio leadership studies for ‘Consideration’ and ‘Initiating Structure’ which appear to cover people and task aspects respectively.
In line with Barrick & Prengler (2018). Kurz (2023) related both to the universal values circumplex of Schwarz (1992) to create a four-quadrant model and developed the Personality Factors range of questionnaires to operationalise Cybernetic Great 8 Theory that offers opportunities to enhance assessments in several ways.

‘RESPONSE STYLE’ is a score based on the sum of all positive qualities that approximates the General Factor of Personality (GFP) which Van der Linden et al (2017) considered a factor of Social Effectiveness that is very closely related to Emotional Intelligence.

In addition to normative reporting, Intra-Personal scores based on an item-level variation of Bartram (1996) generate reports that are ipsatised thus resolving the normative-ipsative debate.

‘STABILITY’ and ‘PLASTICITY’ scores are displayed, and their difference interpreted as a contrast between dependability and proactivity. Extreme difference scores are considered ‘out-of-balance’ and indicative of increased derailment risk'.

‘TASK’ and ‘PEOPLE’ scores are displayed, and their difference interpreted with the aid of paradoxes based on a quasi-circumplex arrangement of Success Factors.

Modular general (PF16), professional (PF32) and executive (PF48) versions cater for different job levels.

Psychometric data on the development sample (N=466) and case study vignettes will be presented. Parallels and differences between Big 5 and Great 8 approaches, and links to the Periodic Table of Personality will be discussed.

ABP Conference, 2024

Rainer has been at the forefront of Computer Based Assessment since 1986 when in his first year P... more Rainer has been at the forefront of Computer Based Assessment since 1986 when in his first year Psychology project he tortured friends with an audio-visual reaction time test he programmed on his Commodore C64.

Studying at the University of Hull under Prof Dave Bartram he secured his first job at SHL and proceeded to explore rule-based item generation, open answer format and speed-accuracy trade-odd in his MSc project.

At SHL he researched MS-DOS, Windows, and WWW test technology. He developed Competency Potential algorithms, ‘Which Way’ career apps and the Great 8 Competencies (Kurz & Bartram, 2002).

Further milestones involved the development of Saville Consulting Wave. Performance 360 and Swift aptitude tests.

At Cubiks be pushed forward the revolutionary PAPI Dynamic Report and the Horizon solution with dynamic top-tie breaker and ‘nipsatised’ scoring.

Most recently he has been presenting on the Periodic Table of Personality and how Cybernetic Big 5 Theory (DeYoung, 2005) could be improved.

He is a passionate advocate for good assessment practice and has shared very critical views on psychometric assessment issues in the TI newsletter of the International Test Commission and at conferences across Europe.0

Rainer will review developments in the field, explain why he hates multiple regression and share his concerns about artificial intelligence.

ICP Conference in Yokohama, 2016

Ability test construct validation study illustrating concerns that EFPA convergent construct vali... more Ability test construct validation study illustrating concerns that EFPA convergent construct validity minimum value of .55 is set too high at a counter-productive level and that the earlier standard of .45 applied in the 'BPS (1996). Review of Personality Instruments (Level B) For Use in Occupational Settings' is more suitable.

ITC COLLOQUIUM, 2021

Paper in the ITC Colloquium 11th July 2021 Symposium 382: Opportunities and Challenges for Person... more Paper in the ITC Colloquium 11th July 2021 Symposium 382: Opportunities and Challenges for Personality Assessment

ITC COLLOQUIUM, 2021

Paper in the virtual ITC Conference Symposium 'Exploring Career Fit using a Range of Constructs, ... more Paper in the virtual ITC Conference Symposium 'Exploring Career Fit using a Range of Constructs, Taxonomies and Techniques'

Association of Test Publishers (ATP), 2019

This poster outlines development of the 'Horizon' assessment solution initially for the China mar... more This poster outlines development of the 'Horizon' assessment solution initially for the China market with three key innovations in testing. For the PAPI 3 SL N version of the 'Personality and Preference Inventory' a novel questionnaire assessment with 162 questions was developed that dynamically combines natural (normative) responding with intrapersonal (ipsative) scoring and a Top Tie Breaker mechanism that identifies Inhibitors. Personality facets are aggregated through validated algorithms into Competency Predictions within a Capability Model. A Cultural Agility output was developed from Chinese characteristics.

BPS DOP Conference, 2020

Personality Assessment Science & Practice: Mapping Measures to the Periodic Table of Personality ... more Personality Assessment Science & Practice:
Mapping Measures to the Periodic Table of Personality

Symposium at the BPS DOP 2020 Conference

Psychological Assessment at Work

This symposium maps personality questionnaire scales to the Periodic Table of Personality (Woods & Anderson, 2016) using the Trait Descriptive Adjectives (TDA) measure of Goldberg (1992). PCA with Varimax rotation on results for N=1213 professionals and managers were used to create orthogonal regression-based factor scores. The mean age of this group was 46.06 (SD 12.86). 63% of the group identified as Female.

The first two papers are based on a co-validation sub-group (N=128) that completed PAPI 3 SL, HPI, HDS and MVPI questionnaires in addition to TDA.

The first paper is concerned with the construct validation of a Big 5 + Achieving research model operationalised through 30 PAPI scales.

The second paper is concerned with the construct validation of the scales of the Hogan questionnaires.

The third paper maps 10 Lumina Spark Aspects representing both poles of each Big 5 factor and 40 sub-ordinated Qualities with adaptive and maladaptive sub-scores.

Paper 1 Summary:

Kurz (2020). Locating Scales of a Multi-level ‘Big 5 + Achieving’ Measure on the Periodic Table of Personality

The first paper is concerned with the construct validation of a Big 5 + Achieving research model operationalised through 30 PAPI scales with TDA scores (N=128) following the methodology of Woods & Anderson (2016). Scores for 20 PAPI dimensions and 10 Facets were aggregated into six factor scores using integer weights based on a prior large-scale validation study. Factors were paired up into Engaging, Enterprising and Excelling clusters which in turn were aggregated into a Performing Total score. TDA correlations largely were as expected determining construct locations on the Periodic Table of Personality. The model resolves the conundrum that Achieving constructs are variously located within Conscientiousness or Extraversion domains, or even at the opposite end of Agreeableness. The three-cluster model builds on the Three Effectiveness Factors (Kurz, Saville & MacIver, 2009) which in turn were inspired by the Alpha and Beta factors of Digman (1997). The results show strong overlap between the Performing Total based on PAPI and the sum of the TDA factors representing the General Factor of Personality (Musek, 2007). The results provide confirmation for the validity and value of multi-level measurement of personality variables.

Paper 2 Summary:

Groenewald & Kurz (2020). Mapping Hogan Bright Side, Dark Side and Inside Scales to the Periodic Table of Personality.

This paper is concerned with the construct validation of Hogan HPI, HDS and MVPI scale constructs with TDA scores (N=128) following the methodology of Woods & Anderson (2016). Scores for all scales as well as summary scores within and across instruments were mapped to the Periodic Table of Personality.

HPI results closely concur with the relevant TDA Big 5 scores although Learning Approach was largely independent.

HDS scores correlated negatively with Extraversion and Emotional Stability for Moving Away scales, positively with Extraversion and Openness for Moving Against scale and positively with Conscientiousness and Agreeableness for Moving Towards scales.

MVPI scales were less strongly correlated with TDA Big 5 scores but generally followed a pattern compatible with the research of Schwartz (1992) on universal values.

The results shed further light on the nature of bright side, dark side and inside measures by mapping them onto the Periodic Table of Personality.

Paper 3 Summary:

Desson (2020). Mapping Lumina Spark to the Periodic Table of Personality

This paper is concerned with the construct validation of Lumina Spark scale constructs with TDA scores (N=671) following the methodology of Woods & Anderson (2016). Scores for 10 Aspects covering the opposing poles of each Big 5 factor and 40 qualities were correlated with orthogonal TDA scores following Woods & Anderson (2016). Most correlations are in line with expectations although a few deviate somewhat e.g. there is more overlap than expected in that Conscientiousness (convergent thinking) and Openness (divergent thinking) attract opposing constructs rather than showing independence. Introverted and Discipline Driven are ‘factor-pure’ aspects whereas Observing, Intimate, Collaborative, Regard for Others, Focuses Feelings, Resilient and Responsive achieve this status at Qualities level.
Maladaptive sub-scales generally follow the correlation pattern of adaptive sub-scales but vary considerably in their correlation with Emotional Stability.

For the most part, Aspects and Qualities were found to correlate as expected, and thus shows the robust convergent validity between Lumina Spark and the TDA. Furthermore, by conceptualising opposite sectors of the circumplex model as discrete dimensions, we were able to show that these sectors are not necessarily direct opposites of each other, with one being the low form of its opposite, but instead discrete Aspects and Qualities can be mapped to these sectors.

BPS DOP Conference in Stratford-upon-Avon, 2004

Presentation on the development of the 'Great 8' that was originally submitted as part of a sympo... more Presentation on the development of the 'Great 8' that was originally submitted as part of a symposium for the 2003 conference.

EAWOP Congress in Lisbon, 2003

Poster Overview Intelligence Theory Multiple Intelligences (MI) The Differential Reasoning Mode... more Poster Overview

Intelligence Theory
Multiple Intelligences (MI)
The Differential Reasoning Model (DREAM)
Competency Behaviour Survey:
Multiple Intelligences Inter-correlations
Interest Correlates
Personality Correlates
Competency Correlates
The World of Work (WoW) Model
Competency at Work

The Differential Reasoning Model (DREAM)

Review of the ability literature and ‘Occupational Testing’ practice lead to the development of the model (Kurz, 2000)
Model contrasts two categories of ability assessment :
Aptitudes (Reasoning/Learning) vs.
Achievements (Knowledge/Skills)
Model distinguishes general abilities (expected to have high traditional ‘g’ loadings) from distinct abilities (expected to have low traditional ‘g’ loadings) with 2 Sectors each:
General (Educational/Practical) vs.
Distinct (Physical/Social)
The four sectors cover 3 Ability Areas each
General Reasoning Ability ‘R’ defined as ‘unit weight’ average of six reasoning areas: V, N, C, D, S, M

Competency Behaviour Survey

Pilot research study (Briceno, 2002)
Joint project with Goldsmith MSc student
Target sample: >120 SHL staff worldwide
Operationalisation:
.xls Spreadsheet Player
Biodata
Ability self-ratings on 12 Multiple Intelligences
‘Great 8’ Competencies (Kurz & Bartram, 2002)
Collection of existing psychometric data for Construct Validation:
Occupational Personality Questionnaire: Big 5
Career Pathfinder In-depth: RIASEC

Multiple Intelligences in the World of Work

Miles (2000): ‘Emotional intelligence may be better conceptualised as an array of behavioural skills dependent on attributes of both personality and ability.’ ‘ …it is probable that emotional intelligence may possess both construct validity and practical utility as a competency (constellation of behaviours) rather than as a particular trait or ability’
Kurz and Bartram (2002) define competencies as ‘sets of behaviours that are instrumental in the delivery of desired results or outcomes. ‘
Kurz (1999) developed the ‘World of Work’ (WoW) model to map out the elements that underlie ‘Person * Environment Interaction’ at work
Basis for SHL Competency Framework (Kurz & Bartram, 2002)

EAWOP Congress in Lisbon, 2003

On-screen Aptitude Testing @ SHL: The Research Questions SHL has 25 years of experience in uti... more On-screen Aptitude Testing @ SHL: The Research Questions

SHL has 25 years of experience in utilising computers in assessment. Since 1990 three generations of on-screen aptitude tests were developed, and a number of studies conducted to research their statistical properties as well as candidate responses. This poster pulls together data from seven studies to address the following questions:

Q1. Are the on-screen and p&p tests psychometrically equivalent?

Q2. Which administration format do candidates prefer?

Q3. On which version do candidates feel more anxious?

Q4. How do results change with technology advances?

Three Generations of SHL On-Screen Aptitude Tests

International Testing System (1990)
MS-Dos Operating System
Verbal (VMG1) and Numerical Reasoning (NMG1)
in 10 Languages
Expert (1993-)
Windows Operating System
VMG1, NMG1, Basic Checking (CP7.1), Diagrammatic Series (DC3.1), Spatial Reasoning (SIT7), Mechanical Comprehension (MT4.1)
On-line Solutions (1997-)
WWW Administration System using Java Applets
Practice Tests
Access Ability (Verbal & Numerical) for unsupervised ‘Distance Assessment’
Verbal (VMG3) & Numerical (NMG3) Reasoning

On-Screen Test Design

In the computerisation of ‘EXPERT’ lessons learned from the MS-DOS predecessor (see Kurz, Torrijos-Sanchez & Bartram (1994) where incorporated e.g. no on-screen calculator; use of mouse/touch screen instead of keyboard; ergonomic design of displays, controls, instructions and examples; and ‘Failsafe’ operation functionality geared towards (virtual) self-administration (see Kurz, 1998). The aim was to achieve transparency, consistency and acceptability across more than 20 questionnaires and 80 tests used with candidates across the entire working population, in over 20 languages. Evaluations by Kurz (1997) demonstrated high candidate satisfaction with instruction and example design, and favourable responses to the Graphical User Interface features. The development of WWW On-line aptitude tests – whether for Supervised or Unsupervised use, posed a number of additional challenges summarised by Evans (2002) relating to user interface, response mode and system feature design.

Study Designs

Test-Retest across Paper & Pencil and Computer mode:
Identical tests with time gap: Study 1, 3-6
Identical test in single session: Study 2
Parallel tests in single session: Study 7
All studies with 2 experimental groups:
Condition P&P-C: Paper & Pencil followed by Computer
Condition C-P&P: Computer followed by Paper & Pencil
Randomised allocation to conditions (except Study 7)
Study 4 also had P&P-P&P and C-C condition
Evaluation forms
Subjects: University Students Study 1, 2, 3 & 5; A-level Students Study 6 & 7; Employees Study 4
Locations: UK apart from Study 5 (Germany)

Psychometric Equivalence Issues

The results of the 7 studies fully support the findings of Mead & Drasgow (1993) that Power tests transfer well to computer format, and Speed tests are problematic. Average sample size weighted Test-Retest reliabilities reached very satisfactory levels of .75 for Verbal, and .80 for Numerical Reasoning, in line with ordinary p&p Test-Retest reliabilities reported in the MGIB manual. The results of Study 6 suggest that other ‘Power’ tests also transfer well to computer format. The research on the Basic Checking test demonstrated the need for different norms, and suggests that Speed tests may well have slightly different measurement characteristics in computerised format. Interestingly, Agapitou (1993) found slightly higher validities for the computerised version of CP7.1 in the prediction of competencies, and Wheeley (1994) found superior validities for computerised versions of VMG and NMG in the prediction of A-level results.

Conclusions

It is reassuring to know that the popular verbal and numerical reasoning tests can be considered equivalent across modes of administration. This means that current manuals and norms can be retained. For more specialised ‘Power’ tests in the Diagrammatic, Spatial and Mechanical area the limited results so far also suggest equivalence. The research on the Clerical test has highlighted the difficulty of achieving equivalence for Speed tests. New norms are required, and further studies to ascertain any differences in the construct measured, and validity. The candidate responses overall (sample size weighted average) show a clear preference for on-screen tests which is coupled with a slight reduction of test anxiety in that mode. Increased candidate familiarity with computers and technological advances make on-screen assessment the natural test administration medium in the new Millennium.

References

Agapitou, G. (1993). Evaluation of the equivalence of the automated mode of administration and the time limit effect on performance in selection tests. MSc Dissertation. University of Hull.
Aldridge, S. (1994). Psychometric testing and computer based assessment. MSc dissertation. University of East London.
Bartram, D. (1994), Computer-based Assessment. International Review of Industrial and Organizational Psychology, 9, 31-69.
Clausen, M. & Wolf, M. (1995). Vergleich zwischen paper-pencil und computerunterstuetzter Fassung von VMG1 and NMG1. University of Kiel.
Evans, T. (2002). Equivalence between Internet-based and paper-and-pencil cognitive ability tests and factors associated with the test mode effect. MSc dissertation. Birkbeck College.
Kurz, R., Torrijos-Sanchez, M. & Bartram, D. (1994). Computer based Assessment: Equivalence or Superiority? Paper at the BPS Occupational Psychology Conference, Birmingham, UK.
Kurz, R. (1997). Developing ‘Expert’ computerised tests. Paper presented at the Fifth European Congress of Psychology in Dublin.
Kurz, R. (1998). Perspectives on the Design of Computer Based Tests. Paper at the Computers in Psychology Conference, York, UK.
Kurz, R. (2000). The Facets of Occupational Testing: General Reasoning Ability, Residual Aptitudes & Speed-Accuracy Balance. PhD dissertation. Manchester: UMIST.
Mead, A. D. & Drasgow, F. (1993).Equivalence of computerized and paper-and-pencil cognitive ability tests: A Meta-analysis. Psychological Bulletin, 114, 449-458.
Torrijos-Sanchez, M. (1991). Evaluation of the automated from of the MGIB tests. MSc dissertation. University of Hull.
Wheeley, B. (1994). An investigation into the differences between computer administration and paper and pencil administration of occupational ability tests. BSc dissertation. Aston University, Birmingham.

EAWOP Congress in Lisbon, 2003

Performance at Work Competencies have become a powerful tool for assessing the performance of pe... more Performance at Work

Competencies have become a powerful tool for assessing the performance of people at work because
they are firmly linked to observable behaviour.
they give managers and their staff a common language for discussing development and career potential issues,
they provide a way to express the goals, culture and values of the Organization in terms of the behaviour expected of employees.
Kurz and Bartram (2002) define competencies as:
sets of behaviours that are instrumental in the delivery of desired results or outcomes.

Integrating Competencies and Assessment Psychology

Conventional Competency Modelling approach:
Client-specific
Pragmatic
Varied
Local linking of assessments to competencies
SHL Competency Framework:
Universal framework (can be tailored to clients)
Theory-based
Standardised
Clear a-priori predictor-criterion links
Integrating psychometric theory and competencies practice to create scientifically sound assessment solutions

Research Basis of ‘Great 8’ Competencies

Statistical modelling & expert equation validation:
Kurz (1999) validated expert system equations predicting the 16 IMC competencies from 30 personality, verbal and numerical ability scales across 7 IVS studies; Average validities of 0.50 for self ratings and 0.30 for boss ratings
Goto (1999) cross-validated equations on 2 Asian samples with average validities of .25 for boss ratings
Inspection of r-matrices from these studies suggested most of predictor-criterion variance could be accounted for by 8 criterion factors
Content analysis: Mapping of IMC, CCCI and WSCI competencies (16 each) into a 20 dimension model
Logical concordance: expert judgements (Warr, 1999) of OPQ-IMC scale item content overlap
Theoretical: Big 5 Personality Model, General Mental Ability (GMA), Need for Achievement, Need for Control

‘Great 8’ Competency Factors

LEADING & DECIDING Taking Control & Exercising Power
SUPPORTING & CO-OPERATING Showing Agreeableness & Cooperation
INTERACTING & PRESENTING Showing Extraversion & People Orientation
ANALYSING & INTERPRETING Reasoning with Words, Numbers & Diagrams
CREATING & CONCEPTUALISING Showing Openness to New Ideas & Experiences
ORGANISING & EXECUTING Showing Conscientiousness & Compliance
ADAPTING & COPING Showing Emotional Maturity & Stress Tolerance
ENTERPRISING & PERFORMING Pursuing Challenges & Achievements

SIOP Conference, 2003

SIOP Paper 2003 The paper presents a criterion-centric approach to validation. It argues that th... more SIOP Paper 2003

The paper presents a criterion-centric approach to validation. It argues that the workplace behaviors or competencies we are interested in predicting can usefully be defined in terms of eight broad factors: the Great Eight. Evidence from 33 validation studies is reviewed that supports the utility of this structure. Personality-based
predictors of the Great Eight show strong correlations with their respective line manager or supervisor ratings of the eight competencies, while correlations with other competencies (i.e. those that do not match the predictor) average around zero. Ability tests correlate, as predicted, with three of the Great Eight competencies. In combination, ability and personality data yield corrected sample-weighted correlations
ranging from 0.23 to 0.44 for the eight predictor-competency pairs. Canonical correlation between the uncorrected predictor and criterion correlation matrices show that personality and ability measures together share over 40% of the criterion variance (R=0.65). Results are also provided showing how the Great Eight relate to overall
measures of job performance and to ratings of promotability. Finally, comparisons are drawn between using the Great Eight and the Big Five as models for the personality predictor domain.

See also Bartram (2005) JAP article.

BPS Occupational Psychology Conference, 2003

Baron, H., Bartram, D. & Kurz, R. (2003). The Great 8 as a framework for validation research. Pro... more Baron, H., Bartram, D. & Kurz, R. (2003). The Great 8 as a framework for validation research. Proceedings of the BPS Occupational Psychology Conference, 71-74.

BPS DOP Conference, 2003

What sort of validation? Process by which the utility of some of assessment can be demonstrated.... more What sort of validation?

Process by which the utility of some of assessment can be demonstrated.
Utility is related to increasing or enhancing organisational effectiveness.
Conventionally:
Measure validity in terms of predictor-criterion correlations
Typically these are ‘many-to-few’ or ‘many-to-one’ relationships
Assess utility using ‘dollar criterion’

Are we asking the right question?

Conventional approach is predictor-centric.
We ask questions that categorise the world in terms of psychological measurement:
What can we predict with instrument X?
OPQ32, MQ, MGIB,
What is the value of ability tests as predictors of training outcome?
Utility equations based on composite predictor ‘r’ values and dollar criterion estimates

What question should we ask?

To get beyond bland generalisations, need a well-articulated model of the domain of work-place behaviours
We need to ask ‘How do we best predict Y?’
Where ‘Y’ is some meaningful, well-defined and important aspect of workplace behaviour.
Our model of the criterion domain not our model of the predictor domain should drive validation research.
Focus on the predictor domain has resulted in clear well articulated specification of that, but a fuzzy ill-defined specification of the criteria.

Criterion-focus re-defines predictors as measures of competency potential

Our validation methodology needs to distinguish:
Competency potential: Measures of dispositional attributes, personality and emotional intelligence, critical faculties, managerial judgement
Competencies: Behaviours, personal performance, and colleagues’ perceptions
Outcomes: Achievements, impacts on organisation’s business and financial performance, track record etc

Competencies

Competencies have become a powerful tool for assessing the performance of people because
they are firmly linked to observable behaviour.
they give managers and their people a common language for discussing development and career potential issues,
they provide a way to express the goals, culture and values of the Organization in terms of the behaviour expected of employees.
Kurz and Bartram (2002) define competencies as:
sets of behaviours that are instrumental in the delivery of desired results or outcomes.

What is a ‘competency framework’

Lots of people talk about competency frameworks, when what they mean are just collections of competencies.
A framework is an articulated set of relationships
It defines the components of a model.
How those components relate to each other
How they relate to other constructs (performance, personality etc).
It is evidence-based.

Deep and surface structures

The framework specifies the generic ‘deep structure’ of the competency domain
Specific competency models are ‘surface structures’
Measurements of competencies are ‘surface measures’.
‘Construct validity’ of competency measures can vary as a function of:
Measurement method – ratings, exercises etc
Assessor perspective – self, boss, peer, etc

A new validation model

The competency framework provides a common basis for describing both predictor measures and criterion measures
Great 8 predictor scores can be generated from combinations of ability, personality and motivation measures
Great 8 criterion scores can be generated by mapping competency ratings to the competency framework
Validity can then be examined using a one-to-one mapping of Great 8 predictor-criterion pairings.

ICP Congress in Montreal, 1996

Presentation Outline  Computer Based Assessment in Careers Guidance  Study Objectives & Hypothe... more Presentation Outline
 Computer Based Assessment in Careers Guidance
 Study Objectives & Hypothesis
 Sample & Methodology
 Results & Discussion
 Conclusions

Study Objectives
 Research psychometric properties of the AIMS
modules
 Investigate Construct Validity by integrating AIMS
under the headings of the Holland Theory of
Vocational Type
 Gather candidate’s views on instruments and process
 Hypothesis: test results likely to be robust; AIMS
components are likely to correlate in a meaningful way
to the Holland types; candidates are likely to feel
positive about materials used, process and outcome

Sample
 N = 152 UK students and adults in higher education
 75% no test experience
 Mean age 25.34 with SD of 10.39
 37.5% male, 62.5% female

Methodology
 Mouse Practice
 6 Ability Tests (Verbal, Numerical, Diagramming,
Checking, Spatial and Mechanical)
 Advanced Occupational Interest Inventory
 Motivation Questionnaire
 Occupational Personality Questionnaire
 Evaluation Forms

Conclusions
 Strong support for using AIMS in Careers Guidance
 Successful AIMS integration under Holland Types
 Blueprint for “The automated Assessment /
Development Centre”
 Future Research:
- Factor analyse General and / or Management Interest
Inventory to generate Holland Types
- Computerised Versions of GAP
- More universal Profile of Occupational Preferences
- Integrated Ability, Interest, Motivation and Style
EXPERT Systems

Test User Conference in Scarborough, 1999

Paper at the Fourth Test User Conference, Scarborough, 10 June 1999 Presentation Outline  Backg... more Paper at the Fourth Test User Conference, Scarborough, 10 June 1999

Presentation Outline
 Background
 Project Objectives & Hypothesis
 Methodology & Samples
 Results & Discussion
 Conclusions & Outlook

Background:
Competencies
 Definition of Competencies: Multidimensional
Constructs that drive successful behavior patterns
 Competency Constructs (Campbell 1993, Kurz 1999):
Multiply determined by antecedents (Ability, Interest,
Motivation, Style) and determinants (Knowledge, Skill,
Experience, Qualification)
 Competency Requirements: Behavior Requirements
multiply determined by Job Objectives, Tasks, Context
and Culture
 Competency Behaviors: Observed Behavior Patterns
that reflect successful Performance at Work

Background:
Managerial Competencies
 Boyatzis (1982)
 Schroder (1989)
 Sparrow (1993)
 Dulcewics (1992)
 Inventory of Management Competencies (SHL,1992)
 Perspectives in Management Competencies
(SHL,1994)

Background:
Validity of Ability & Personality Variables
 Validity Reviews (e.g. Ghiselli, 1973)
 Validity Generalisation (e.g. Hunter & Schmidt, 1986)
 Meta Analysis of Validity Studies (e.g. Schmitt et
al.,1984; Barrick & Mount, 1989)
 Meta Analysis of 20 SHL Validity Studies (Robertson &
Kinder, 1993)
 International Validation Studies (Gibbons et al., 1995)

Background:
Computer Based Test Interpretation
(CBTI)
 since 1960’ies especially for MMPI and other clinical
instruments
 since 1987 Occupational Personality Questionnaire
(OPQ) Expert System Management Skills Potential
 1990 Work Profiling System: Human Attribute Based
Person-Job Match
 1995 OPQ Competency Based Narrative
 1995 Competency Based Person-Job Match
 1999 DecisionMaker
 1999 OPQ32 vs. IMC Potential

Methodology
 Standardised Validation Studies as pioneered in 4
AMT - IMC Studies (SHL, 1992)
 Concurrent Design
 Predictors:
- Occupational Personality Questionnaire (CM4.2)
- Verbal Critical Reasoning (VMG)
- Numerical Critical Reasoning (NMG)
 Criteria:
- Inventory of Management Competencies (16 scales)
- Normative and Ipsative Sub-scales
- ‘Boss’ & ‘Self’ Ratings

Validation Studies -
Sample Sizes, Country, Industry Sector
 A: N=131, UK, Distribution
 B: N=34, UK, Banking
 C: N=90, UK, Financial Services
 D: N=114, UK, Paper Manufacturing
 E: N=68, UK, Banking
 F: N=103, USA, Rubber Manufacturing
 G:N=503, Turkey, Glass Manufacturing

Designing Prediction Equations
 Statistical Analysis: First order and multiple
correlations between predictors and criteria (within
and across studies)
 Expert Judgement: A-priori hypothesised relations
 Social Judgement: Logical Concordance (Warr, 1997):
 Content Validity: Justifiable from Job Analysis
 Measurement: Exclusion of Artefacts (Ipsativity, Halo)
 Inclusion of Scales meeting all Criteria with Unit
Weights of 1, 2 ,3 or 4

Results:
Validity
 Validation of Weighted Composite Prediction Scores
 Predictions are overall stable & valid across jobs,
organisations and cultures for most competencies
 Good validities for Ability (Median around .25; up to
.47)
 High Validities for Personality variables vs. ‘Boss’
(Median around .3; up to .52)
 High Validities for Personality variables vs. ‘Self’’
(Median around .5; up to .76)

Conclusions & Outlook
 Highly successful Research & Development Project
 Impressive Validity Evidence
 Competency Based P-J Match now in use world-wide
 Competency Based OPQ Narrative displaying &
verbalising Prediction Scores under Preparation
 Future Research & Development:
- Integration of other predictors
- Integration under different categories
- User friendly report design
- Documentation

European Congress of Psychology, 2001

Poster at the European Congress of Psychology Dr. Rainer H. Kurz, SHL Group plc London, 2 July 2001

Kogan Page Book , 2008

Book Chapter

A book chapter outlining the origin of the Great 8 Competencies and the 'World of Work' (WoW) model.

International Congress of Psychology (ICP) in Prague, 2024

Discussant: Dragos Iliescu, University of Bucharest

Intelligent Testing Systems: Past, Present and Future
Rainer Hermann Kurz, PhD
HUCAMA Analytics, London

Computer-Based Assessment across Personality, Ability and Competency Factors
Michele Guarini
HUCAMA Group, Copenhagen

Personality Factors:

The modular range features general, professional, and executive level versions with 80, 160 and 240 questions measuring 16, 32 and 48 facets respectively grouped into 8 factors.

Ability Factors:

Competency Factors:

Sociomapping and Team Profile Analyzer in Psychometric Assessment
Pauline Willis
Lauriate, Australia

Insights in Motion: A Comprehensive Model for Tracking Employee Experience Over Time
Richard T. Justenhoven, PhD
Welliba, Germany

International Congress of Psychology, 2024

In addition to normative reporting, Intra-Personal scores based on an item-level variation of Bartram (1996) generate reports that are ipsatised thus resolving the normative-ipsative debate.

‘TASK’ and ‘PEOPLE’ scores are displayed, and their difference interpreted with the aid of paradoxes based on a quasi-circumplex arrangement of Success Factors.

Modular general (PF16), professional (PF32) and executive (PF48) versions cater for different job levels.

ABP Conference, 2024

At SHL he researched MS-DOS, Windows, and WWW test technology. He developed Competency Potential algorithms, ‘Which Way’ career apps and the Great 8 Competencies (Kurz & Bartram, 2002).

Further milestones involved the development of Saville Consulting Wave. Performance 360 and Swift aptitude tests.

At Cubiks be pushed forward the revolutionary PAPI Dynamic Report and the Horizon solution with dynamic top-tie breaker and ‘nipsatised’ scoring.

Most recently he has been presenting on the Periodic Table of Personality and how Cybernetic Big 5 Theory (DeYoung, 2005) could be improved.

Rainer will review developments in the field, explain why he hates multiple regression and share his concerns about artificial intelligence.

ICP Conference in Yokohama, 2016

ITC COLLOQUIUM, 2021

Association of Test Publishers (ATP), 2019

BPS DOP Conference, 2020

Symposium at the BPS DOP 2020 Conference

Psychological Assessment at Work

The first two papers are based on a co-validation sub-group (N=128) that completed PAPI 3 SL, HPI, HDS and MVPI questionnaires in addition to TDA.

The first paper is concerned with the construct validation of a Big 5 + Achieving research model operationalised through 30 PAPI scales.

The second paper is concerned with the construct validation of the scales of the Hogan questionnaires.

The third paper maps 10 Lumina Spark Aspects representing both poles of each Big 5 factor and 40 sub-ordinated Qualities with adaptive and maladaptive sub-scores.

Paper 1 Summary:

Kurz (2020). Locating Scales of a Multi-level ‘Big 5 + Achieving’ Measure on the Periodic Table of Personality

Paper 2 Summary:

Groenewald & Kurz (2020). Mapping Hogan Bright Side, Dark Side and Inside Scales to the Periodic Table of Personality.

HPI results closely concur with the relevant TDA Big 5 scores although Learning Approach was largely independent.

MVPI scales were less strongly correlated with TDA Big 5 scores but generally followed a pattern compatible with the research of Schwartz (1992) on universal values.

The results shed further light on the nature of bright side, dark side and inside measures by mapping them onto the Periodic Table of Personality.

Paper 3 Summary:

Desson (2020). Mapping Lumina Spark to the Periodic Table of Personality

BPS DOP Conference in Stratford-upon-Avon, 2004

EAWOP Congress in Lisbon, 2003

Poster Overview Intelligence Theory Multiple Intelligences (MI) The Differential Reasoning Mode... more Poster Overview

The Differential Reasoning Model (DREAM)

Competency Behaviour Survey

Multiple Intelligences in the World of Work

EAWOP Congress in Lisbon, 2003

On-screen Aptitude Testing @ SHL: The Research Questions SHL has 25 years of experience in uti... more On-screen Aptitude Testing @ SHL: The Research Questions

Q1. Are the on-screen and p&p tests psychometrically equivalent?

Q2. Which administration format do candidates prefer?

Q3. On which version do candidates feel more anxious?

Q4. How do results change with technology advances?

Three Generations of SHL On-Screen Aptitude Tests

On-Screen Test Design

Study Designs

Psychometric Equivalence Issues

Conclusions

References

EAWOP Congress in Lisbon, 2003

Performance at Work Competencies have become a powerful tool for assessing the performance of pe... more Performance at Work

Integrating Competencies and Assessment Psychology

Research Basis of ‘Great 8’ Competencies

‘Great 8’ Competency Factors

SIOP Conference, 2003

SIOP Paper 2003 The paper presents a criterion-centric approach to validation. It argues that th... more SIOP Paper 2003

See also Bartram (2005) JAP article.

BPS Occupational Psychology Conference, 2003

BPS DOP Conference, 2003

What sort of validation? Process by which the utility of some of assessment can be demonstrated.... more What sort of validation?

Are we asking the right question?

What question should we ask?

Criterion-focus re-defines predictors as measures of competency potential

Competencies

What is a ‘competency framework’

Deep and surface structures

A new validation model

ICP Congress in Montreal, 1996

Sample
 N = 152 UK students and adults in higher education
 75% no test experience
 Mean age 25.34 with SD of 10.39
 37.5% male, 62.5% female

Test User Conference in Scarborough, 1999

Paper at the Fourth Test User Conference, Scarborough, 10 June 1999 Presentation Outline  Backg... more Paper at the Fourth Test User Conference, Scarborough, 10 June 1999

Presentation Outline
 Background
 Project Objectives & Hypothesis
 Methodology & Samples
 Results & Discussion
 Conclusions & Outlook

European Congress of Psychology, 2001

Poster at the European Congress of Psychology Dr. Rainer H. Kurz, SHL Group plc London, 2 July 2001

BPS DOP Conference, 2002

Occupational ability tests are widely used for selection, development and guidance applications i... more Occupational ability tests are widely used for selection, development and guidance applications in occupational psychology. Their developers and users tend to only consider the ‘Raw’ or ‘Right’ score, i.e. the number of questions answered correctly, and disregard information about speed and accuracy. This paper proposes that validity increments can be gained from alternative approaches to ability test scoring and outlines a theoretical framework model. An empirical study on n=208 subjects across six ability tests shows the distribution of the theory based test scores, and demonstrates construct as well as criterion-related validity. Finally a practical example illustrates the issues at hand and their significance for assessment practice.

ITC Conference in Liverpool, 2008

This practical paper is based on a skills session as delivered at the BPS Special Group in Coachi... more This practical paper is based on a skills session as delivered at the BPS Special Group in Coaching Psychology Conference held 2007 in London. The first part of our paper provides a review of the extant research evidence on 360 degree feedback at with focus on effects on individual development, making explicit links to the implications for coaching practice throughout. We conclude that 360 degree feedback is primarily effective when conceptualised and utilised as a finely grained means of instigating individual behaviour change and learning on job relevant attributes and facilitated by a skilled feedback giver. This provides a clear rationale for its use in coaching. We outline how an actual profile can be used as part of a coaching session, using the Saville Work Wave ® Performance 360 as an example.

Assessment and Development Matters

The Ministry of Defence introduced a new assessment process for entry into the Senior Civil Servi... more The Ministry of Defence introduced a new assessment process for entry into the Senior Civil Service (SCS) in 2008, including psychometric testing for the first time. This paper addresses the verbal and numerical ability test data from 2008 (paper-and-pencil) and 2009 (online). The tests are highly reliable both in terms of individual annual administration and overall. Feedback from candidates and assessors showed they also had high face-validity regarding content and utility. The descriptive statistics indicate that the sample was much better than the Professionals and Graduates norm group and also better – especially on the numerical test – than an SCS (unpublished) norm group. Women slightly outperformed men on the verbal test and men clearly outperformed females on the numerical test. Overall, both men and women faired better on the verbal than the numerical test. Whilst younger candidates scored better on the tests, both the younger and older groups scored better than the norm group. There was some indication that candidates who were invited to attend assessment Stage 3 prioritised accuracy over speed compared to those who were not invited. The differences in test scores between online and paper completion were negligible. Authors: Antonia Dietmann, Amanda Feggetter & Rainer Kurz

Organizational Effectiveness

... is carried out in a particular industrial sector, and leads to culture-based competencies tha... more ... is carried out in a particular industrial sector, and leads to culture-based competencies that reflect ... Gotoh (1999) successfully cross-validated these prediction equations with a Japanese sample. ... PersonEnvironment models (Mischel & Shoda, 1995) lend themselves to a vi-sual ...

Proceedings of the British Psychological Society …, 2003

Baron, H., Bartram, D. & Kurz, R. (2003). The Great 8 as a framework for validation resea... more Baron, H., Bartram, D. & Kurz, R. (2003). The Great 8 as a framework for validation research. Proceedings of the BPS Occupational Psychology Conference, 71-74.

SIOP Paper 2003 The paper presents a criterion-centric approach to validation. It argues that the... more SIOP Paper 2003 The paper presents a criterion-centric approach to validation. It argues that the workplace behaviors or competencies we are interested in predicting can usefully be defined in terms of eight broad factors: the Great Eight. Evidence from 33 validation studies is reviewed that supports the utility of this structure. Personality-based predictors of the Great Eight show strong correlations with their respective line manager or supervisor ratings of the eight competencies, while correlations with other competencies (i.e. those that do not match the predictor) average around zero. Ability tests correlate, as predicted, with three of the Great Eight competencies. In combination, ability and personality data yield corrected sample-weighted correlations ranging from 0.23 to 0.44 for the eight predictor-competency pairs. Canonical correlation between the uncorrected predictor and criterion correlation matrices show that personality and ability measures together share over 40% of the criterion variance (R=0.65). Results are also provided showing how the Great Eight relate to overall measures of job performance and to ratings of promotability. Finally, comparisons are drawn between using the Great Eight and the Big Five as models for the personality predictor domain. See also Bartram (2005) JAP article.

The Coaching Psychologist, Dec 1, 2008

This brief article is a direct and invited rejoinder to the comments by David Bartram on our orig... more This brief article is a direct and invited rejoinder to the comments by David Bartram on our original paper on the use of 360 degree feedback in coaching. Bartram raises important issues which are likely to be of concern and interest to coaches and other users of psychometric tools, even if they do not relate directly to the primary focus of our original paper. This rejoinder reiterates our original approach which was based on a practical workshop on coaching that included a case study reinforcing the benefits of good structural alignment and validity between psychometric trait measures and criterion measures of work effectiveness. This we consider a logical and practical extension of the criterion-centric perspective to measurement, which is a measurement perspective we share with Bartram.Next, our rejoinder addresses how the Great Eight model mapping was used in the analysis as a unit weighted aggregate to predict overall effectiveness at work rather than predict behavioural competencies. Overall effectiveness (and ability) is central to Saville Consulting Wave® Performance 360 criterion measure, and we put this in the context of a hypothetical gap analysis to make the link back to the application in coaching.High level models which lead to aggregation of both predictors and criterion variables are increasingly commonplace in the literature and include Bartram (2005). We further this work by defining a reliable overall (global) effectiveness scale and reporting validities against the total scale source as well as its de-aggregated items each of which represents a distinct segment of overall performance. Rather than differentially weighting these predictors we used a priori unit weighting of the Great Eight predictors from Wave® Professional Styles and OPQ32i. We were primarily concerned with testing the hypothesis that the results were non-zero in predicting overall effectiveness in the original article.We confirm that Wave® Professional Styles trait measure of Competency Potential clearly improved on OPQ32i’s prediction of overall effectiveness (p<.05, two tailed, N=169) in this study.Finally, we emphasise the advantages of single co-validation studies in the comparison of the validity of different models and psychometric tools based on fully pre-hypothesised equations to aggregate predictor scales. This approach, paired with other methodologies, we argue will lead to the scientific advancement of the field.

Assessment and Development Matters

This article investigates a new hierarchical model of work effectiveness that integrates ability,... more This article investigates a new hierarchical model of work effectiveness that integrates ability, personality and competency assessment models with overall measures of work effectiveness. A total of 308 participants rated their effectiveness on the Saville Consulting Wave® Performance 360 measure, and were also rated by an external rater. Results suggest an overarching Effectiveness factor that subdivides into Promoting Change, Demonstrating Capability and Working Together components. Authors: Rainer Kurz, Peter Saville and Rab MacIver

Introduction General Mental Ability (GMA) is according to meta-analytical research the single bes... more Introduction General Mental Ability (GMA) is according to meta-analytical research the single best predictor of work performance. The question arises how speed and accuracy in answering questions across various ability areas underpin test performance and validity. Objective The present study investigates the psychometric of sub-scores of a short measure of GMA. Method/Design Psychometric studies were undertaken involving Logiks General which measures GMA through 24 Verbal, 16 Numerical and 10 Abstract problems in three separately timed sections of 4 minutes. Results Internal Consistency reliability on N=177 was satisfactory for the Total score (.76) and slightly below the .70 level required for stand-alone interpretation for Verbal (.64), Numerical (.62) and Abstract (.57) sub-tests. Same day completions (N=138) resulted in an adequate correlation of .766 for Total, .801 for Speed and .757 for Verbal sub-scores. Accuracy (.681), Numerical (.660) and Abstract (.599) fell slightly sho...

Introduction This symposium aims to further the understanding of personality structure and improv... more Introduction This symposium aims to further the understanding of personality structure and improve applied assessment practices. It builds on the well-established Big 5 model of personality factors offering pioneering insights into higher-order models as well as possible extensions through motivational factors. The symposium offers pioneering advances to theoretical understanding and practical measurement of personality variables using straightforward techniques. Mapping Personality Inventories to the Periodic Table of Personality: Impact of Non-orthogonality Introduction How can personality inventory scales be organized to a coherent cross-inventory framework? Woods & Anderson (2015) addressed these questions by mapping PI scales to the Abridged Five Dimensional Circumplex (AB5C) Model (Hofstee, De Raad, & Goldberg, 1992), resulting in a Periodic Table of Personality Traits. The resultant structural framework was designed explicitly to enable researchers and others to continue to build knowledge and understanding of personality trait structure and the impact of personality traits in work psychology theory and practice. Objectives This paper elaborates the mapping methodology applied in the study and explores the application of the methodology in a small scale cross-validation study using Big 5 domain rather than orthogonal scores. Design/Methodology The paper first sets out the conceptual band empirical rationale for the mapping of the construction of the Periodic Table followed by a study where N=65 managers and administrators completed the 20 scales of PAPI 2 alongside TDA-100 for which no item data was available so that TDA domain scores were used instead introducing Non-orthogonality issues. Results Three key steps are elaborated: a) extraction of orthogonal representations of the Big Five from Goldberg’s TDA-100 trait markers (Goldberg, 1992); b) correlation of PI scales with the resultant factors; c) classification of scales based on primary and secondary loadings. In the cross-validation scales with primary correlations below .40 were unstable. Secondary correlations were generally unstable as factors were not orthogonal. Vector values >= .40 retained their primary loading. Conclusions By understanding the structural properties of the PI, researchers can integrate their theorizing with results and findings compiled using different inventories and add in a more substantive and coherent way to the literature on personality at work. Cross-validation studies can assist in this effort but may be misleading if non-orthogonal data is used. Circumplex Personality Factor Structure Investigation of 8 Aspects and 24 Qualities in the Lumina Spark Mandala Introduction I n the light of renewed interest in Circumplex models this paper explores the factor structure of the Lumina Spark personality questionnaire that measures the four bipolar constructs of popular Type measures (such as MBTI) through 8 separate Aspects thus covering Extraversion, Agreeableness, Openness and Conscientiousness. Each Aspect is measured through 3 distinct qualities. Objectives This paper explores whether the circumplex structure implied by the Lumina Spark Mandala can be reproduced through factor analysis. Design/Methodology Questionnaire data from an international group of professionals and managers (N=2158) was subjected to PCA. Results The first two PCA components accounted for 40% and 21% of the variance. In a rotated solution the first factor emerged as a Task factor contrasting Conscientiousness with Openness while the second factor emerged as a People factor contrasting Extraversion with Agreeableness. The Component Plot in Rotated Space accurately represented the Mandala graph that underpins Lumina Spark reporting. The correlation between opposing Aspect scales averaged -.737 as expected whereas the perimeter correlations averaged .345. Within the ‘Alpha’ themes of Agreeableness and Conscientiousness the perimeter values averaged .414, within ‘Beta’ themes .423 but dropped at the border between the two to only .124. I n an extraction of 24 Qualities the first component accounted for 31% and the second for 18% of the variance. The two dimensional mapping of the 24 qualities closely corresponded to the actual arrangement on the Mandala but suggesting that on this particular sample Reliable & Structured as well as Measured & Observed would be better represented if swapped over. Conclusions The results clearly confirm the circumplex nature of the Lumina Spark Mandala. They also lend support to the notion of distinct Task and People axes that interact with Alpha and Beta higher-order personality factors to define four distinct clusters of personality. Validation-centric Application of Linear Mixed Effects Models using Higher-Order and Facet Personality Constructs Introduction Inspired by Kurz & Woods (2015) this paper explores a proprietary Big 5 measure with 24 scales through GFP, Alpha/Beta and variations of…

International Journal of Selection and Assessment, 1996

… Coaching Psychology Review, 2007

This practical paper is based on a skills session as delivered at the first International Coachin... more This practical paper is based on a skills session as delivered at the first International Coaching Psychology Conference held in 2006. It commences by discussing the use of psychometrics in general by emphasising the four psychometric principles as hallmarks of ...

The Psychometric Forum (London), 2020

Virtual Presentation at The Psychometric Forum (London): 'Applications of the Periodic Table of... more Virtual Presentation at The Psychometric Forum (London):

'Applications of the Periodic Table of Personality: Mapping Hogan Insight Series (HPI Bright Side, HDS Dark Side and MVPI Inside) vs. HUCAMA Personality and Competency Factors'

Validation of Saville Consulting Wave and Swift Analysis Aptitude in Mauritius Rainer Kurz;Hennie... more Validation of Saville Consulting Wave and Swift Analysis Aptitude in Mauritius Rainer Kurz;Hennie Kriek;Nailah Moussa;Fred Guest; International Journal of Psychology