Miloš Kankaraš | UNESCO - Academia.edu (original) (raw)

Books by Miloš Kankaraš

Research paper thumbnail of Assessment of Social and Emotional Skills in the OECD's SSES Study

Draft2Digital, 2023

In the world of educational and psychological assessment, few endeavors are as challenging or as ... more In the world of educational and psychological assessment, few endeavors are as challenging or as rewarding as the development of instruments that accurately measure social and emotional skills. The OECD's Study on Social and Emotional Skills (SSES) represents a monumental step in this field, with the creation of the SSES Inventory – an instrument crafted through a global, multidisciplinary effort. This manuscript delves into the intricate process and the significant outcomes of this landmark study, as detailed in the comprehensive account of its development.
At the heart of the SSES study lies a rigorous instrument development process, tailored to assess 15 social and emotional skills across 45 scales. This expansive project marshaled the collective expertise of 15 international research teams and engaged more than 150,000 participants—students, parents, and teachers. Its reach spanned eleven cities globally, underscoring the universal importance of these skills. With the coordination of over a thousand professionals, including project managers, researchers, translators, and data analysts, the process was a mosaic of cultural and methodological diversity, aiming for an assessment tool of unparalleled scope and precision.
Initially grappling with 380 assessment items, the teams' methodical analysis and meticulous refinement culminated in the selection of the most robust 120 items, some unaltered and others slightly adapted. The multi-faceted approach merged quantitative and qualitative research methodologies, yielding a harmonious synthesis of data and insights. This approach ensured that the resulting scales were not only reliable and valid but also resonant across different cultures, languages, and educational contexts.
The book sheds light on the SSES Inventory's performance along the various stages of its empirical testing. The instrument demonstrated remarkable versatility, adapting to the developmental stages of students beginning from the age of ten. One of the key highlights was the Inventory's triangulated approach, utilizing feedback from students, parents, and teachers to enrich the data and enhance scale validity.

Research paper thumbnail of Essays on Measurement Equivalence in Cross-Cultural Survey Research: A Latent Class Approach.

Research paper thumbnail of Testing for Measurement Invariance With Latent Class Analysis

In E. Davidov, P. Schmidt, & J. Billiet (Eds.), Cross-cultural analysis. Methods and applications, 2010

Testing for measurement invariance can be done within the context of multigroup latent class anal... more Testing for measurement invariance can be done within the context of multigroup latent class analysis. Latent class analysis can model any type of discrete level data, which makes it an obvious choice when nominal indicators are used or when a researcher's aim is at classifying respondents in latent classes. The multigroup latent class (LC) model can be specified in three different ways, i.e. by adopting a probabilistic, a log-linear or a logistic parameterization. We define and compare these different forms of parameterization. The starting point is the standard LC model in which indicators and latent variables are defined at the nominal level. Additionally, we focus on LC models with ordinal indicators as well as LC factor models with ordinal indicators. Testing for measurement invariance involves estimating LC models with different degrees of homogeneity. We explain the procedure for investigating measurement invariance at both the scale as well as the item level. We illustrate the approach with two examples. The first example is a multigroup LC analysis with nominal indicators; the second a multigroup LC factor analysis with ordinal indicators.

Papers by Miloš Kankaraš

Research paper thumbnail of Editorial: Theory and empirical practice in research on social and emotional skills

Frontiers in Education, 2022

Research on social and emotional skills is multi-disciplinary, combining a unique mix of academic... more Research on social and emotional skills is multi-disciplinary, combining a unique mix of academic and practice-oriented research approaches. It features both theoretical and empirical research streams, with lively discussions on the conceptual foundation and structure of these skills, their origin, development, malleability, and relevance. In this Research Topic, we have tried to reflect the diversity of the field’s perspectives, disciplines, and methodological approaches and its unique blend of research and policy. The Research Topic contains empirical and theoretical works representing both streams of research on social and emotional skills. It also presents both academic and practice-oriented studies on a variety of topics in the field. Still, studies presented in this Research Topic place a particular emphasis on the issue of the application of gathered academic knowledge in educational settings. Selected studies do so either through examining current states, developing policy interventions, or evaluating their effectiveness.

Research paper thumbnail of Kankaraš, M. (2021) Workplace learning Determinants and Consequences

Workplace Learning: Determinants and Consequences, 2021

Workplace learning is a critical tool for both employees and organisations in their continuous ad... more Workplace learning is a critical tool for both employees and organisations in their continuous adjustment to various global trends reshaping our workplaces, economies and societies. Workers learn most when engaging in new and challenging activities in which they are granted sufficient autonomy to solve issues themselves or in collaboration with their colleagues or mentors. Findings based on the data from the 2019 European Company Survey indicate that informal learning is the key aspect of workplaces in terms of company performance and employee wellbeing. Such results call for more attention from policy-makers and organisations’ management to various forms of informal learning at the workplace. This is particularly the case when considering the apparent lack of beneficial effects of formal and non-formal learning on employee wellbeing and company performance. These findings illustrate the overwhelming prominence of informal learning at the workplace compared to formal and non-formal learning forms. They also indicate the often untapped learning potential that exists at workplaces. However, analyses show that most companies are still not offering optimal learning environments and opportunities to their workers. Such a situation hurts both employees and companies but also presents an opportunity for action at organisational, national and international levels. Such an action would aim to create institutional conditions, legal frameworks, learning resources, and incentives for companies to start organising their workplaces in a way that will facilitate workplace learning, especially its various informal forms.

Research paper thumbnail of Skills Matter: Further Results from the Survey of Adult Skills. OECD Skills Studies

Research paper thumbnail of Social and Emotional Skills for Student Success and Well-Being: Conceptual Framework for the OECD Study on Social and Emotional Skills. OECD Education Working Papers, No. 173

Research paper thumbnail of Heterogeneity in solidarity attitudes in Europe. In-sights from a multiple-group latent-class factor ap-proach

Research paper thumbnail of Attendance in early childhood education and care programmes and academic proficiencies at age 15

OECD Publishing, 2020

Early years are a critical period for skill development. In this sense, the Early Childhood Edu... more Early years are a critical period for skill development. In this sense, the Early Childhood Education and Care (ECEC) programmes have an important role in promoting children’s learning during this period. This study aimed to explore the effectiveness of ECEC programmes by analysing the relationship between students’ ECEC attendance and their later academic proficiency using PISA 2015 data. PISA results show that across the OECD countries, students who had attended ECEC tend to have higher scores in academic proficiencies at the age of 15. However, these differences in academic proficiencies between those who attended ECEC versus those who did not attend are almost nil when students’ socio-economic status (SES) is considered. This relationship reflects differential access to learning opportunities for children from deprived contexts. Furthermore, results show that entering ECEC programmes earlier than the typical time is associated with lower proficiencies at the age of 15. Therefore, earlier entry to ECEC is not necessarily beneficial.

Learning benefits of ECEC provision vary considerably across PISA countries illustrating the importance of a country-specific policy context and the quality of their ECEC provision. The analyses of several quality indicators point out that the improved quality of ECEC programmes is associated with higher academic skills at later stages. These results highlight that mere attendance to ECEC programmes is not enough to ensure better academic performance. The quality of the educational provision, especially concerning those students from disadvantaged backgrounds, should be ensured.

Research paper thumbnail of Assessing students' social and emotional skills through triangulation of assessment methods

OECD Publishing, 2019

Triangulation – a combined use of different assessment methods or sources to evaluate psychologic... more Triangulation – a combined use of different assessment methods or sources to evaluate psychological constructs – is still a rarely used assessment approach in spite of its potential in overcoming inherent constraints of individual assessment methods. This paper uses field test data from a new OECD Study on Social and Emotional Skills to examine the triangulated assessment of 19 social and emotional skills of 10- and 15-year-old students across 11 cities and countries. This study assesses students’ social and emotional skills combining three sources of information: students’ self-reports and reports by parents and teachers. We examine convergent and divergent validities of the assessment scales and the analytical value of combining information from multiple informants. Findings show that students’, parents’ and teachers’ reports on students’ skills overlap to a substantial degree. In addition, a strong ‘common rater’ effect is identified for all three informants and seems to be reduced when we use the triangulation approach. Finally, triangulation provides skill estimates with stronger relations to various life outcomes compared with individual student, parent or teacher reports.

Research paper thumbnail of Assessment framework of the OECD Study on Social and Emotional Skills

OECD Publishing, 2019

The OECD’s new Study on Social and Emotional Skills aims to provide policy makers, educators, fam... more The OECD’s new Study on Social and Emotional Skills aims to provide policy makers, educators, families and communities with a comprehensive set of tools to foster students’ social and emotional learning. The Study’s assessment framework – presented in this paper – is a result of an extensive literature review of previous research, existing frameworks and assessment approaches in the area of social and emotional skills. The framework, developed by a team of experts in various domains, incorporates evidence from psychology, education, organisational sciences, sociology, economy, and public policy. This framework describes the objectives, characteristics and expected outcomes of the Study. It presents the conceptual model of social and emotional skills assessed in the Study, their development, malleability and predictive value. The framework also discusses how factors in students’ family, school and peer environment influence their social and emotional skills’ development along with the contextual questionnaires designed to gather this information. The framework also presents the Study’s design, assessment approach, instrument development process, sampling procedures and data collection methods.

Research paper thumbnail of The Power and Promise of Early Learning

OECD Publishing, 2018

Early learning matters and can be strengthened and supported through contexts where children spen... more Early learning matters and can be strengthened and supported through contexts where children spend much of their time. Strong early learning experiences provide young children with opportunities to enjoy exploring their own interests and growing capabilities. Strong early learning also positively predicts well-being across a range of indicators in adulthood, including general well-being, physical and mental health, educational attainment and employment. Areas of early learning that are of particular importance include: language and literacy; numeracy and other non-verbal cognitive skills; self-regulation; emotional health, social well-being and social and emotional skills. These domains are interrelated, meaning that holistic assessment of a range of skills is necessary to understand well-being in early childhood and its implications for the future. Assessment of early learning provides an essential opportunity to reflect on whether governments, communities, schools and families are supporting the power and promise of early learning as intended.

Research paper thumbnail of Social and emotional skills for student success and well-being: Conceptual framework for the OECD study on social and emotional skills

OECD Publishing, 2018

In an increasingly fast-changing, complex and diverse world, social and emotional skills are beco... more In an increasingly fast-changing, complex and diverse world, social and emotional skills are becoming ever more important. In this paper we present an overview of literature on social and emotional skills, describing the nature and structure of these skills, their development, malleability and factors that influence them, their cross-cultural comparability and their relevance for a wide range of educational, economic and life outcomes. The paper also represents a conceptual framework for the OECD’s new Study on Social and Emotional Skills, an international survey that assesses 10- and 15-year-old students in a number of cities and countries around the world.
We focus on the underlying skills within and outside of the widely researched Big Five model that are found to be more predictive and policy relevant. We examine the relationships of these skills with a variety of indicators of individual and societal wellbeing such as education, employment and income, health, and personal well-being. The paper discusses the structure of child’s social and emotional skills and the developmental trajectories of these skills across a lifetime. It presents the evidence of malleability of these skills as well as their relevance across a wide range of cultural contexts.

Research paper thumbnail of Researching Measurement Equivalence In Cross-Cultural Studies

Psihologija, 2010

In cross-cultural comparative studies it is essential to establish equivalent measurement of rele... more In cross-cultural comparative studies it is essential to establish equivalent measurement of relevant constructs across cultures. If this equivalence is not confirmed it is difficult if not impossible to make meaningful comparison of results across countries. This work presents concept of measurement equivalence, its relationship with other related concepts, different equivalence levels and causes of inequivalence in cross-cultural research. It also reviews three main approaches to the analysis of measurement equivalence – multigroup confirmatory factor analysis, differential item functioning, and multigroup latent class analysis – with special emphasis on their similarities and differences, as well as comparative advantages.

Research paper thumbnail of Cross-National and Cross-Ethnic Differences in Attitudes: A Case of Luxembourg

Cross-Cultural Research, Jun 2012

Minorities’ attitudes can be compared to attitudes of fellow citizen within the host country as w... more Minorities’ attitudes can be compared to attitudes of fellow citizen within the host country as well as to attitudes of the motherland. Given the heterogeneity of Luxembourg’s minority groups, this country is a relevant example case in which the comparison needs to involve answering a two-folded question. First we analyze the level of measurement equivalence, i.e. the extent to which different groups can be compared. Secondly, we examine whether ethnic-cultural groups within Luxembourg resemble citizens from their native country more than their country of residence. Using EVS-data from 2008 we demonstrate different types of outcomes. Results indicate that cultural background is more important than national context in the case of culturally more distant minorities to Luxembourg’s resident population, and that national setting is the prevailing factor when minorities are from neighboring countries. The effect of a common national setting is also important with regards to the issue of measurement equivalence, where it contributes to greater comparability of intra-national, cross-ethnic comparisons.

Research paper thumbnail of Measurement Equivalence of Ordinal Items: A Comparison of Factor Analytic, Item Response Theory, and Latent Class Approaches

Sociological Methods and Research, 2011

Three distinctive methods of assessing measurement equivalence of ordinal items, i.e. confirmator... more Three distinctive methods of assessing measurement equivalence of ordinal items, i.e. confirmatory factor analysis, differential item functioning using item response theory and latent class factor analysis, make different modeling assumptions and adopt different procedures. Simulation data are used to compare the performance of these three approaches in detecting the sources of measurement inequivalence. For this purpose, we simulated Likert-type data using two non-linear models, one with categorical and one with continuous latent variables. Inequivalence was set up in the slope parameters (loadings) as well as in the item intercept parameters in a form resembling agreement and extreme response styles. Results indicate that the item response theory and latent class factor models can relatively accurately detect and locate inequivalence in the intercept and slope parameters both at the scale and the item level. Confirmatory factor analysis performs well when inequivalence is located in the slope parameters, but wrongfully indicates inequivalence in the slope parameters when inequivalence is located in the intercept parameters. Influences of sample size, number of inequivalent items in a scale, and model fit criteria on the performance of the three methods are also analysed.

Research paper thumbnail of Analysis of Cross-Cultural Comparability of PISA 2009 Scores

Journal of Cross-Cultural Psychology, 2013

The Programme for International Student Assessment (PISA) is a large-scale cross-national study t... more The Programme for International Student Assessment (PISA) is a large-scale cross-national study that measures academic competences of 15-year old students in mathematics, reading, and science in more than 50 countries/economies around the world. PISA results are usually aggregated and presented in so called “league tables” in which countries are compared and ranked in each of the three scales. However, in order to compare results obtained from different groups/countries one must first be sure that the tests measure the same competences in all cultures. In this paper this is tested by examining the level of measurement equivalence in the 2009 PISA dataset using an Item Response Theory approach (IRT) and analyzing Differential Item Functioning (DIF). Measurement in-equivalence was found in the form of uniform DIF. In-equivalence occurred in a majority of test questions in all three scales researched and is, on average, of moderate size. It varies considerably both across items and across countries. When this uniform DIF is accounted for in the in-equivalent model, resulting country scores change considerably in the cases of the ‘Mathematics’, ‘Science’ and especially ‘Reading’ scale. These changes tend to occur simultaneously and in the same direction in groups of regional countries. The most affected seems to be Southeast Asian countries/territories whose scores, although among the highest in the initial, homogeneous model, additionally increase when accounting for in-equivalence in the scales.

Research paper thumbnail of Simultaneous Latent Class Analysis Across Groups

In A. C. Michalos (Ed.), Encyclopedia of Quality of Life Research. Springer., 2014

Simultaneous latent-class analysis across groups (SLCAG) is an extension of the standard latent c... more Simultaneous latent-class analysis across groups (SLCAG) is an extension of the standard latent class (LC) model for the examination of measurement equivalence/invariance. It can be used to compare the latent structure derived from a set of discrete item responses between multiple groups, e.g., between males and females, between Japanese, Americans and Dutch, young and old, and between ill and healthy. Whereas the more commonly used multigroup confirmatory factor analysis (MCFA) assumes that the underlying latent variables are continuous, SLCAG treats the latent variables either as nominal – e.g. to identify a typological classification from a given set of categorical indicators – or as ordinal – e.g. to investigate the scalability of a set of categorical indicators. These two specifications are sometimes referred as LC cluster and LC factor models, respectively.

Research paper thumbnail of Measurement Equivalence and Extreme Response Bias In the Comparison of Attitudes Across Europe

Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 2008

It is generally accepted that both measurement inequivalence and extreme response bias can seriou... more It is generally accepted that both measurement inequivalence and extreme response bias can seriously distort measurement of attitudes and subsequent causal models. However, these two issues have rarely been investigated together. In this article we demonstrate the flexibility of a multigroup latent class factor approach in both analysing measurement equivalence and detecting extreme response bias. Using data from the European Value Survey from 1999/2000, we identified an extreme response bias in answering Likert type questions on attitudes towards morals of compatriots. Furthermore, we found measurement inequivalence in form of direct effects of countries on response variables. When only one of these two issues – either measurement inequivalence or extreme response bias - was included into measurement model estimated effects of countries on attitudinal dimension were different from those obtained with a model that includes both measurement issues. Using this all-inclusive model we have got more valid estimates of the differences between countries on measured attitude.

Research paper thumbnail of Measurement Equivalence In Solidarity Attitudes In Europe: Insights From a Multiple-Group Latent-Class Factor Approach

International Sociology, Jan 1, 2009

Comparing solidarity attitudes of European citizens is highly relevant in the context of European... more Comparing solidarity attitudes of European citizens is highly relevant in the context of European integration and unification. Such comparisons, however, are only valid if responses to attitude questions reflect true differences in solidarity and, hence, the measurement of latent solidarity attitudes is comparable. Often comparability is assumed, rarely is it tested. This research presents a multiple-group latent-class factor analysis of a set of questions concerning solidarity towards different social groups, taken from the 1999/2000 wave of the European Value Study. This multiple-group comparison reveals that equivalence in attitude measurement is not established straightforwardly. However, once the sources of measurement inequivalence are taken into account, valid comparison between countries is possible. It turns out that European countries are less divided on the solidarity attitudes than would have been concluded from a model that did not account for heterogeneity in measurement. At the same time, the improved model only slightly altered country rankings on solidarity dimensions.

Research paper thumbnail of Assessment of Social and Emotional Skills in the OECD's SSES Study

Draft2Digital, 2023

In the world of educational and psychological assessment, few endeavors are as challenging or as ... more In the world of educational and psychological assessment, few endeavors are as challenging or as rewarding as the development of instruments that accurately measure social and emotional skills. The OECD's Study on Social and Emotional Skills (SSES) represents a monumental step in this field, with the creation of the SSES Inventory – an instrument crafted through a global, multidisciplinary effort. This manuscript delves into the intricate process and the significant outcomes of this landmark study, as detailed in the comprehensive account of its development.
At the heart of the SSES study lies a rigorous instrument development process, tailored to assess 15 social and emotional skills across 45 scales. This expansive project marshaled the collective expertise of 15 international research teams and engaged more than 150,000 participants—students, parents, and teachers. Its reach spanned eleven cities globally, underscoring the universal importance of these skills. With the coordination of over a thousand professionals, including project managers, researchers, translators, and data analysts, the process was a mosaic of cultural and methodological diversity, aiming for an assessment tool of unparalleled scope and precision.
Initially grappling with 380 assessment items, the teams' methodical analysis and meticulous refinement culminated in the selection of the most robust 120 items, some unaltered and others slightly adapted. The multi-faceted approach merged quantitative and qualitative research methodologies, yielding a harmonious synthesis of data and insights. This approach ensured that the resulting scales were not only reliable and valid but also resonant across different cultures, languages, and educational contexts.
The book sheds light on the SSES Inventory's performance along the various stages of its empirical testing. The instrument demonstrated remarkable versatility, adapting to the developmental stages of students beginning from the age of ten. One of the key highlights was the Inventory's triangulated approach, utilizing feedback from students, parents, and teachers to enrich the data and enhance scale validity.

Research paper thumbnail of Essays on Measurement Equivalence in Cross-Cultural Survey Research: A Latent Class Approach.

Research paper thumbnail of Testing for Measurement Invariance With Latent Class Analysis

In E. Davidov, P. Schmidt, & J. Billiet (Eds.), Cross-cultural analysis. Methods and applications, 2010

Testing for measurement invariance can be done within the context of multigroup latent class anal... more Testing for measurement invariance can be done within the context of multigroup latent class analysis. Latent class analysis can model any type of discrete level data, which makes it an obvious choice when nominal indicators are used or when a researcher's aim is at classifying respondents in latent classes. The multigroup latent class (LC) model can be specified in three different ways, i.e. by adopting a probabilistic, a log-linear or a logistic parameterization. We define and compare these different forms of parameterization. The starting point is the standard LC model in which indicators and latent variables are defined at the nominal level. Additionally, we focus on LC models with ordinal indicators as well as LC factor models with ordinal indicators. Testing for measurement invariance involves estimating LC models with different degrees of homogeneity. We explain the procedure for investigating measurement invariance at both the scale as well as the item level. We illustrate the approach with two examples. The first example is a multigroup LC analysis with nominal indicators; the second a multigroup LC factor analysis with ordinal indicators.

Research paper thumbnail of Editorial: Theory and empirical practice in research on social and emotional skills

Frontiers in Education, 2022

Research on social and emotional skills is multi-disciplinary, combining a unique mix of academic... more Research on social and emotional skills is multi-disciplinary, combining a unique mix of academic and practice-oriented research approaches. It features both theoretical and empirical research streams, with lively discussions on the conceptual foundation and structure of these skills, their origin, development, malleability, and relevance. In this Research Topic, we have tried to reflect the diversity of the field’s perspectives, disciplines, and methodological approaches and its unique blend of research and policy. The Research Topic contains empirical and theoretical works representing both streams of research on social and emotional skills. It also presents both academic and practice-oriented studies on a variety of topics in the field. Still, studies presented in this Research Topic place a particular emphasis on the issue of the application of gathered academic knowledge in educational settings. Selected studies do so either through examining current states, developing policy interventions, or evaluating their effectiveness.

Research paper thumbnail of Kankaraš, M. (2021) Workplace learning Determinants and Consequences

Workplace Learning: Determinants and Consequences, 2021

Workplace learning is a critical tool for both employees and organisations in their continuous ad... more Workplace learning is a critical tool for both employees and organisations in their continuous adjustment to various global trends reshaping our workplaces, economies and societies. Workers learn most when engaging in new and challenging activities in which they are granted sufficient autonomy to solve issues themselves or in collaboration with their colleagues or mentors. Findings based on the data from the 2019 European Company Survey indicate that informal learning is the key aspect of workplaces in terms of company performance and employee wellbeing. Such results call for more attention from policy-makers and organisations’ management to various forms of informal learning at the workplace. This is particularly the case when considering the apparent lack of beneficial effects of formal and non-formal learning on employee wellbeing and company performance. These findings illustrate the overwhelming prominence of informal learning at the workplace compared to formal and non-formal learning forms. They also indicate the often untapped learning potential that exists at workplaces. However, analyses show that most companies are still not offering optimal learning environments and opportunities to their workers. Such a situation hurts both employees and companies but also presents an opportunity for action at organisational, national and international levels. Such an action would aim to create institutional conditions, legal frameworks, learning resources, and incentives for companies to start organising their workplaces in a way that will facilitate workplace learning, especially its various informal forms.

Research paper thumbnail of Skills Matter: Further Results from the Survey of Adult Skills. OECD Skills Studies

Research paper thumbnail of Social and Emotional Skills for Student Success and Well-Being: Conceptual Framework for the OECD Study on Social and Emotional Skills. OECD Education Working Papers, No. 173

Research paper thumbnail of Heterogeneity in solidarity attitudes in Europe. In-sights from a multiple-group latent-class factor ap-proach

Research paper thumbnail of Attendance in early childhood education and care programmes and academic proficiencies at age 15

OECD Publishing, 2020

Early years are a critical period for skill development. In this sense, the Early Childhood Edu... more Early years are a critical period for skill development. In this sense, the Early Childhood Education and Care (ECEC) programmes have an important role in promoting children’s learning during this period. This study aimed to explore the effectiveness of ECEC programmes by analysing the relationship between students’ ECEC attendance and their later academic proficiency using PISA 2015 data. PISA results show that across the OECD countries, students who had attended ECEC tend to have higher scores in academic proficiencies at the age of 15. However, these differences in academic proficiencies between those who attended ECEC versus those who did not attend are almost nil when students’ socio-economic status (SES) is considered. This relationship reflects differential access to learning opportunities for children from deprived contexts. Furthermore, results show that entering ECEC programmes earlier than the typical time is associated with lower proficiencies at the age of 15. Therefore, earlier entry to ECEC is not necessarily beneficial.

Learning benefits of ECEC provision vary considerably across PISA countries illustrating the importance of a country-specific policy context and the quality of their ECEC provision. The analyses of several quality indicators point out that the improved quality of ECEC programmes is associated with higher academic skills at later stages. These results highlight that mere attendance to ECEC programmes is not enough to ensure better academic performance. The quality of the educational provision, especially concerning those students from disadvantaged backgrounds, should be ensured.

Research paper thumbnail of Assessing students' social and emotional skills through triangulation of assessment methods

OECD Publishing, 2019

Triangulation – a combined use of different assessment methods or sources to evaluate psychologic... more Triangulation – a combined use of different assessment methods or sources to evaluate psychological constructs – is still a rarely used assessment approach in spite of its potential in overcoming inherent constraints of individual assessment methods. This paper uses field test data from a new OECD Study on Social and Emotional Skills to examine the triangulated assessment of 19 social and emotional skills of 10- and 15-year-old students across 11 cities and countries. This study assesses students’ social and emotional skills combining three sources of information: students’ self-reports and reports by parents and teachers. We examine convergent and divergent validities of the assessment scales and the analytical value of combining information from multiple informants. Findings show that students’, parents’ and teachers’ reports on students’ skills overlap to a substantial degree. In addition, a strong ‘common rater’ effect is identified for all three informants and seems to be reduced when we use the triangulation approach. Finally, triangulation provides skill estimates with stronger relations to various life outcomes compared with individual student, parent or teacher reports.

Research paper thumbnail of Assessment framework of the OECD Study on Social and Emotional Skills

OECD Publishing, 2019

The OECD’s new Study on Social and Emotional Skills aims to provide policy makers, educators, fam... more The OECD’s new Study on Social and Emotional Skills aims to provide policy makers, educators, families and communities with a comprehensive set of tools to foster students’ social and emotional learning. The Study’s assessment framework – presented in this paper – is a result of an extensive literature review of previous research, existing frameworks and assessment approaches in the area of social and emotional skills. The framework, developed by a team of experts in various domains, incorporates evidence from psychology, education, organisational sciences, sociology, economy, and public policy. This framework describes the objectives, characteristics and expected outcomes of the Study. It presents the conceptual model of social and emotional skills assessed in the Study, their development, malleability and predictive value. The framework also discusses how factors in students’ family, school and peer environment influence their social and emotional skills’ development along with the contextual questionnaires designed to gather this information. The framework also presents the Study’s design, assessment approach, instrument development process, sampling procedures and data collection methods.

Research paper thumbnail of The Power and Promise of Early Learning

OECD Publishing, 2018

Early learning matters and can be strengthened and supported through contexts where children spen... more Early learning matters and can be strengthened and supported through contexts where children spend much of their time. Strong early learning experiences provide young children with opportunities to enjoy exploring their own interests and growing capabilities. Strong early learning also positively predicts well-being across a range of indicators in adulthood, including general well-being, physical and mental health, educational attainment and employment. Areas of early learning that are of particular importance include: language and literacy; numeracy and other non-verbal cognitive skills; self-regulation; emotional health, social well-being and social and emotional skills. These domains are interrelated, meaning that holistic assessment of a range of skills is necessary to understand well-being in early childhood and its implications for the future. Assessment of early learning provides an essential opportunity to reflect on whether governments, communities, schools and families are supporting the power and promise of early learning as intended.

Research paper thumbnail of Social and emotional skills for student success and well-being: Conceptual framework for the OECD study on social and emotional skills

OECD Publishing, 2018

In an increasingly fast-changing, complex and diverse world, social and emotional skills are beco... more In an increasingly fast-changing, complex and diverse world, social and emotional skills are becoming ever more important. In this paper we present an overview of literature on social and emotional skills, describing the nature and structure of these skills, their development, malleability and factors that influence them, their cross-cultural comparability and their relevance for a wide range of educational, economic and life outcomes. The paper also represents a conceptual framework for the OECD’s new Study on Social and Emotional Skills, an international survey that assesses 10- and 15-year-old students in a number of cities and countries around the world.
We focus on the underlying skills within and outside of the widely researched Big Five model that are found to be more predictive and policy relevant. We examine the relationships of these skills with a variety of indicators of individual and societal wellbeing such as education, employment and income, health, and personal well-being. The paper discusses the structure of child’s social and emotional skills and the developmental trajectories of these skills across a lifetime. It presents the evidence of malleability of these skills as well as their relevance across a wide range of cultural contexts.

Research paper thumbnail of Researching Measurement Equivalence In Cross-Cultural Studies

Psihologija, 2010

In cross-cultural comparative studies it is essential to establish equivalent measurement of rele... more In cross-cultural comparative studies it is essential to establish equivalent measurement of relevant constructs across cultures. If this equivalence is not confirmed it is difficult if not impossible to make meaningful comparison of results across countries. This work presents concept of measurement equivalence, its relationship with other related concepts, different equivalence levels and causes of inequivalence in cross-cultural research. It also reviews three main approaches to the analysis of measurement equivalence – multigroup confirmatory factor analysis, differential item functioning, and multigroup latent class analysis – with special emphasis on their similarities and differences, as well as comparative advantages.

Research paper thumbnail of Cross-National and Cross-Ethnic Differences in Attitudes: A Case of Luxembourg

Cross-Cultural Research, Jun 2012

Minorities’ attitudes can be compared to attitudes of fellow citizen within the host country as w... more Minorities’ attitudes can be compared to attitudes of fellow citizen within the host country as well as to attitudes of the motherland. Given the heterogeneity of Luxembourg’s minority groups, this country is a relevant example case in which the comparison needs to involve answering a two-folded question. First we analyze the level of measurement equivalence, i.e. the extent to which different groups can be compared. Secondly, we examine whether ethnic-cultural groups within Luxembourg resemble citizens from their native country more than their country of residence. Using EVS-data from 2008 we demonstrate different types of outcomes. Results indicate that cultural background is more important than national context in the case of culturally more distant minorities to Luxembourg’s resident population, and that national setting is the prevailing factor when minorities are from neighboring countries. The effect of a common national setting is also important with regards to the issue of measurement equivalence, where it contributes to greater comparability of intra-national, cross-ethnic comparisons.

Research paper thumbnail of Measurement Equivalence of Ordinal Items: A Comparison of Factor Analytic, Item Response Theory, and Latent Class Approaches

Sociological Methods and Research, 2011

Three distinctive methods of assessing measurement equivalence of ordinal items, i.e. confirmator... more Three distinctive methods of assessing measurement equivalence of ordinal items, i.e. confirmatory factor analysis, differential item functioning using item response theory and latent class factor analysis, make different modeling assumptions and adopt different procedures. Simulation data are used to compare the performance of these three approaches in detecting the sources of measurement inequivalence. For this purpose, we simulated Likert-type data using two non-linear models, one with categorical and one with continuous latent variables. Inequivalence was set up in the slope parameters (loadings) as well as in the item intercept parameters in a form resembling agreement and extreme response styles. Results indicate that the item response theory and latent class factor models can relatively accurately detect and locate inequivalence in the intercept and slope parameters both at the scale and the item level. Confirmatory factor analysis performs well when inequivalence is located in the slope parameters, but wrongfully indicates inequivalence in the slope parameters when inequivalence is located in the intercept parameters. Influences of sample size, number of inequivalent items in a scale, and model fit criteria on the performance of the three methods are also analysed.

Research paper thumbnail of Analysis of Cross-Cultural Comparability of PISA 2009 Scores

Journal of Cross-Cultural Psychology, 2013

The Programme for International Student Assessment (PISA) is a large-scale cross-national study t... more The Programme for International Student Assessment (PISA) is a large-scale cross-national study that measures academic competences of 15-year old students in mathematics, reading, and science in more than 50 countries/economies around the world. PISA results are usually aggregated and presented in so called “league tables” in which countries are compared and ranked in each of the three scales. However, in order to compare results obtained from different groups/countries one must first be sure that the tests measure the same competences in all cultures. In this paper this is tested by examining the level of measurement equivalence in the 2009 PISA dataset using an Item Response Theory approach (IRT) and analyzing Differential Item Functioning (DIF). Measurement in-equivalence was found in the form of uniform DIF. In-equivalence occurred in a majority of test questions in all three scales researched and is, on average, of moderate size. It varies considerably both across items and across countries. When this uniform DIF is accounted for in the in-equivalent model, resulting country scores change considerably in the cases of the ‘Mathematics’, ‘Science’ and especially ‘Reading’ scale. These changes tend to occur simultaneously and in the same direction in groups of regional countries. The most affected seems to be Southeast Asian countries/territories whose scores, although among the highest in the initial, homogeneous model, additionally increase when accounting for in-equivalence in the scales.

Research paper thumbnail of Simultaneous Latent Class Analysis Across Groups

In A. C. Michalos (Ed.), Encyclopedia of Quality of Life Research. Springer., 2014

Simultaneous latent-class analysis across groups (SLCAG) is an extension of the standard latent c... more Simultaneous latent-class analysis across groups (SLCAG) is an extension of the standard latent class (LC) model for the examination of measurement equivalence/invariance. It can be used to compare the latent structure derived from a set of discrete item responses between multiple groups, e.g., between males and females, between Japanese, Americans and Dutch, young and old, and between ill and healthy. Whereas the more commonly used multigroup confirmatory factor analysis (MCFA) assumes that the underlying latent variables are continuous, SLCAG treats the latent variables either as nominal – e.g. to identify a typological classification from a given set of categorical indicators – or as ordinal – e.g. to investigate the scalability of a set of categorical indicators. These two specifications are sometimes referred as LC cluster and LC factor models, respectively.

Research paper thumbnail of Measurement Equivalence and Extreme Response Bias In the Comparison of Attitudes Across Europe

Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 2008

It is generally accepted that both measurement inequivalence and extreme response bias can seriou... more It is generally accepted that both measurement inequivalence and extreme response bias can seriously distort measurement of attitudes and subsequent causal models. However, these two issues have rarely been investigated together. In this article we demonstrate the flexibility of a multigroup latent class factor approach in both analysing measurement equivalence and detecting extreme response bias. Using data from the European Value Survey from 1999/2000, we identified an extreme response bias in answering Likert type questions on attitudes towards morals of compatriots. Furthermore, we found measurement inequivalence in form of direct effects of countries on response variables. When only one of these two issues – either measurement inequivalence or extreme response bias - was included into measurement model estimated effects of countries on attitudinal dimension were different from those obtained with a model that includes both measurement issues. Using this all-inclusive model we have got more valid estimates of the differences between countries on measured attitude.

Research paper thumbnail of Measurement Equivalence In Solidarity Attitudes In Europe: Insights From a Multiple-Group Latent-Class Factor Approach

International Sociology, Jan 1, 2009

Comparing solidarity attitudes of European citizens is highly relevant in the context of European... more Comparing solidarity attitudes of European citizens is highly relevant in the context of European integration and unification. Such comparisons, however, are only valid if responses to attitude questions reflect true differences in solidarity and, hence, the measurement of latent solidarity attitudes is comparable. Often comparability is assumed, rarely is it tested. This research presents a multiple-group latent-class factor analysis of a set of questions concerning solidarity towards different social groups, taken from the 1999/2000 wave of the European Value Study. This multiple-group comparison reveals that equivalence in attitude measurement is not established straightforwardly. However, once the sources of measurement inequivalence are taken into account, valid comparison between countries is possible. It turns out that European countries are less divided on the solidarity attitudes than would have been concluded from a model that did not account for heterogeneity in measurement. At the same time, the improved model only slightly altered country rankings on solidarity dimensions.

Research paper thumbnail of Implicit theories of creativity: Cross-cultural study

Psihologija, 2009

In this work we analyzed laypersons' implicit theories of creativity in two different national cu... more In this work we analyzed laypersons' implicit theories of creativity in two different national cultures: Serbia (N=257) and the United States of America (N=255). Relying on previous works in this field, we have constructed a questionnaire comprised of 52 indicative and 36 contra-indicative personal characteristic which were rated by respondents on a 5-point scale on criteria of their creativity and desirability. Results show that both groups have similar conceptions of a creative person, which they see as an energetic, self-confident individual, gifted with creative talents and exceptional intellectual abilities, with profound emotionality and brightness. The main difference between the two groups is that respondents from Serbia, contrary to their American counterparts, do not perceive characteristics which reflect obedience to social norms as a contra-indication to creativity. Respondents have mostly seen creative attributes as desirable, although there is a number of characteristics that are rated differently in terms of their creativity and desirability. These results confirm that creativity and desirability are two distinct concepts and indicate positive view of creativity as a phenomenon and creative person as such.

Research paper thumbnail of Metacognition - A New Cognitive Paradigm

Psihologija, 2004

This article reviews concept of metacognition, defined as: (a) knowledge about ones own cognitive... more This article reviews concept of metacognition, defined as: (a) knowledge about ones own cognitive activity, (b) strategies to monitor and regulate cognitive activity and behavior, and (c) subjective or metacognitive experiences which comes from some changes or temporary difficulties in cognitive functioning. While describing different conceptualizations of metacognition, its development, fields of application, relation with intelligence, and its constrictions and ambiguity, we attempt to present new and emerging metacognitive paradigm, which is, for a relatively short period, succeeded to improve, expand, and redefine wide range of theoretical and practical fields in psychology, on new and original way. How do we become conscious of our own cognitive processes? What role and significance that consciousness has, what is the functional level above thinking processes and how that level, which monitor and control cognitive activity, works. Metacognition is concept that presents, as so far, the most important insight in those human mind areas, which, although very important, remained on the margin of psychological investigations until now.

Research paper thumbnail of Domestic Violence Interventions: Effectiveness of Intervention Programs

Gender Equality #4, 2022

Domestic violence was long ignored or underestimated policy topic. It was not until the last deca... more Domestic violence was long ignored or underestimated policy topic. It was not until the last decade of the 20th century that most countries started focusing on this issue and introducing lawful protections against it in their legal systems. Increased attention to the problem resulted in a growing number of intervention programs aiming to prevent the occurrence or reoccurrence of various forms of domestic violence. Unfortunately, in the beginning, these interventions mainly relied on anecdotal accounts and untested assumptions. However, this is slowly changing in recent years, with an increasing number of programs using more robust empirical methods to evaluate their desired impacts.
Without valid and reliable data on the effectiveness of domestic violence interventions, we will not be able to introduce positive change in this area. That is why this book focuses on examining and evaluating existing empirical evidence from around the world on the effectiveness of interventions in the field of domestic violence. In other words, we try to answer “what works”, i.e. what is known to be an effective intervention strategy against domestic violence, under which conditions, and for which outcomes.

Research paper thumbnail of Violence Against Women and Girls: Effectiveness of Intervention Programs

Gender Equality #3, 2022

The widespread prevalence of physical, sexual and psychological abuse and violence that affect ma... more The widespread prevalence of physical, sexual and psychological abuse and violence that affect many women and girls was a largely overlooked topic of policy interest for a long time. Fortunately, this has started to change over the last two decades after a consistent effort of women-rights and aligned non-governmental groups at local, national and international levels.
With the attention of policymakers on the issue now largely secured, what is needed is a collection of solid empirical evidence on the effectiveness of various intervention programs that aim to reduce, eliminate or prevent violence against women and girls (VAWG). Such empirical evidence would help inform and guide the formulation and implementation of effective support programs, policy actions and prevention strategies.
Unfortunately, until recently, most empirical research on this topic was based on anecdotal accounts or exploratory studies administered on non-representative samples. However, the situation is gradually improving in recent years with the administration of a growing number of empirical studies on issues related to violence against women and girls (VAWG). Some of these studies have also evaluated their impact, offering clues about the effectiveness of various intervention approaches in preventing different forms of VAWG.
This report aims to overview and synthesise existing empirical evidence from impact assessment studies that have evaluated intervention programs in the area of VAWG. Namely, this report tries to answer “what works” and “what doesn’t work” in VAWG interventions, but also “what is still unknown” regarding the effectiveness of such interventions. The evidence is collected from both high-income and low- and middle-income countries and includes findings across different types of VAWG and intervention approaches.

Research paper thumbnail of The Global State of Gender Equality: An Overview of Empirical Findings

Gender Equality #2, 2022

Despite the persistent data gaps, a large amount of empirical evidence on the state of gender equ... more Despite the persistent data gaps, a large amount of empirical evidence on the state of gender equality has been gathered worldwide, especially during the last few decades. This international data-gathering activity over the last decade largely revolved around the UN's 2030 Sustainable Development Goals (SDGs). Still, many other global, regional, and national initiatives and empirical programs have also contributed to the pool of available gender data. This report represents a concise overview of the key aspects of the available empirical evidence on the state of gender equality worldwide. The report starts with an overview of the status of the SDGs in terms of gender equality. It follows with a short outline of the main results across available global gender equality indicators and measures and other important global empirical programs dealing with this topic. The report also discusses the observed data inadequacies and gaps and the possible ways of improving the quantity and quality of gender data.

Research paper thumbnail of Policy and Research on Gender Equality: An Overview

Gender Equality #1, 2022

This report provides a brief and up-to-date overview of the global state of gender equality polic... more This report provides a brief and up-to-date overview of the global state of gender equality policy and research. In the first part, dealing primarily with policy, the report lists the main actors and stakeholders in the field and outlines the development of international legal and policy frameworks on gender equality and related issues. The report also introduces key frameworks and perspectives on gender equality and briefly discusses the main aspects and dimensions identified across various approaches. In the second part, dealing with empirical issues, the author examines the main sources of empirical evidence, international analytical studies and indicators, and data depositories on gender equality. Finally, the report concludes with the chapter outlining key limitations and empirical data gaps in internationally available empirical evidence.

Research paper thumbnail of Personality Matters: Relevance and Assessment of Personality Characteristics

Personality characteristics shape human behaviour and influence a wide range of life events and o... more Personality characteristics shape human behaviour and influence a wide range of life events and outcomes. They do so not only through their direct effects on life outcomes, but also through their indirect
effects on other important personal factors and intermediate life events, such as the development of cognitive capacities, the attainment of educational qualifications and the formation of a family. As such,
personality characteristics have a demonstrable relevance for a wide range of policy issues and represent an important, although often neglected, subject of policy interest.
This paper reviews the scientific literature covering a wide range of personality characteristics, discussing their conceptualizations and main features, their relevance for important outcomes in life and
work, and the chief ways they are measured. It aims to provide a comprehensive overview of various attributes of personality from the perspective of their potential importance for the Survey of Adult Skills
(PIAAC), taking into account their analytical potential and policy relevance. The paper also outlines and evaluates the most important measurement instruments for each personality characteristic, with a focus on short self-report scales as the most appropriate form for inclusion in large-scale international surveys. Finally, it presents some considerations related to the evaluation and promotion of personality
characteristics and introduces the substantive and measurement criteria that could be used to select the personality attributes, and related measurement scales, to include in large-scale surveys.

Research paper thumbnail of Skills Matter: Further Results From the Survey of Adult Skills

The capacity to manage information and solve problems using computers is becoming a necessity as ... more The capacity to manage information and solve problems using computers is becoming a necessity as ICT applications permeate the workplace, the classroom and lecture hall, the home, and social interaction more generally. The Survey of Adult Skills, a product of the OECD Programme for the International Assessment of Adult Competencies (PIAAC), was designed to measure adults’ proficiency in several key information-processing skills, namely literacy, numeracy and problem solving in technology-rich environments. Adults who are highly proficient in the skills measured by the survey are likely to be able to make the most of the opportunities created by the technological and structural changes modern societies are going through. Those who struggle to use new technologies are at greater risk of losing out.
The results from the first round of the survey, covering 24 countries and economies, were reported in the OECD Skills Outlook 2013: First Results from the Survey of Adult Skills. Another nine countries and economies collected data during 2014-15. This report presents the main findings for all 33 countries and economies that participated in the study over the two rounds. It finds substantial variation across countries/economies in adults’ average proficiency in the three domains assessed. More than 80 score points separate the highest- and lowest-scoring countries in literacy and numeracy, although many countries and economies score within a relatively close range of each other. Within countries and economies, proficiency scores in literacy and numeracy vary considerably: on average, 62 score points separate the 25% of adults who attained the highest and lowest scores in literacy; in numeracy, 68 score points separate those two groups.
In almost all countries/economies, a sizeable proportion of adults (18.5% of adults, on average) has poor reading skills and poor numeracy skills (22.7% of adults, on average). Around one in four adults has no or only limited experience with computers or lacks confidence in their ability to use computers. In addition, nearly one in two adults is proficient only at or below Level 1 in problem solving in technology-rich environments. This adult can only use familiar applications to solve problems that involve few steps and explicit criteria, such as sorting e-mails into pre-existing folders.

Research paper thumbnail of Skills Matter - Country Note: Slovenia

 On average, adults in Slovenia score below the OECD average in literacy, numeracy and problem s... more  On average, adults in Slovenia score below the OECD average in literacy, numeracy and problem solving in technology-rich environments.
 Around one in four adults in Slovenia has poor literacy, numeracy and problem-solving skills, a slightly larger proportion than the OECD average.
 Adults' literacy scores have improved substantially over the past two decades.
 The variation in Slovenian adults' proficiency scores is greater than on average across OECD countries.
 Differences in proficiency related to age, education and social background are considerable among adults in Slovenia.
 Most workers in Slovenia are well-matched with their jobs.
 In Slovenia, as in other OECD countries, higher proficiency in literacy and numeracy has a positive impact on labour force participation and wages, and on several non-economic outcomes, such as trust in others, political efficacy, participation in volunteer activities and self-reported health.

Research paper thumbnail of Skills Matter - Country Note: New Zealand

 Adults in New Zealand score above the OECD average in literacy, numeracy and problem solving in... more  Adults in New Zealand score above the OECD average in literacy, numeracy and problem solving in technology-rich environments.
 New Zealand's immigrant population is one of the most skilled across OECD countries. At the same time, native-born New Zealanders who speak English as a second language are over-represented among adults with low proficiency.
 The differences in skills proficiency related to age, gender, education and social background are less pronounced in New Zealand than in other countries. However, sharp ethnic differences, particularly for Māori and Pacific peoples, exist in New Zealand.
 In New Zealand, even more so than in other OECD countries, higher proficiency in literacy and numeracy have a positive impact on labour force participation and wages.
 The relationship between literacy and levels of trust in others, political efficacy, participation in volunteer activities and self-reported health is positive and mostly in line with those observed in other OECD countries.

Research paper thumbnail of Overview Report of the 2013 European Company Survey - Workplace Practices: Patterns, Performance and Well-Being

This report, based on the 2013 edition of the European Company Survey, provides policymakers and ... more This report, based on the 2013 edition of the European Company Survey, provides policymakers and practitioners with comprehensive information on workplace practices in terms of work organisation, human resource management, direct participation and social dialogue. As well as documenting the incidence of these practices, the report shows how practices relate to each other and to the outcomes for companies and workers. The underlying thesis is that implementing certain combinations of workplace practices can have a positive effect on outcomes for both workers and companies.
The ECS 2013 was carried out at a time when Europe was only just starting to recover from the biggest economic crisis to hit the world since the Great Depression of the 1930s. For many companies, therefore, the crisis is likely to have had a major influence on the experiences which they report. While the survey was not specifically designed to capture the effects of the crisis, the findings do need to be interpreted with this context in mind.

Research paper thumbnail of The Future of Work in Europe: Job Quality and Work Organisation for a Smart, Sustainable and Inclusive Growth

Employment Outlook 2014, 2015

This chapter assesses the future EU labour market challenges and opportunities in terms of job qu... more This chapter assesses the future EU labour market challenges and opportunities in terms of job quality and work organisation and their likely impact on labour market developments over the next 10 years.
It presents recent developments in job quality and work organisation (and their interactions) and highlights their impact on productivity, labour market participation and social cohesion as indicated by recent research. It then explores how technological progress and innovation, globalisation, demographic change and the greening of the economy may affect the workforce’s potential via their impact on job quality and work organisation. It ends by discussing how labour market policies can help prevent, cushion or correct adverse developments in job quality and work organisation associated with those structural changes, including issues such as polarisation and inequality, while reinforcing positive developments.