Mengxiao Zhu - Academia.edu (original) (raw)
Papers by Mengxiao Zhu
Communications in computer and information science, Dec 31, 2022
Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Aug 14, 2022
Cognitive diagnostic assessment is a fundamental task in intelligent education, which aims at qua... more Cognitive diagnostic assessment is a fundamental task in intelligent education, which aims at quantifying students' cognitive level on knowledge attributes. Since there exists learning dependency among knowledge attributes, it is crucial for cognitive diagnosis models (CDMs) to incorporate attribute hierarchy when assessing students. The attribute hierarchy is only explored by a few CDMs such as Attribute Hierarchy Method, and there are still two significant limitations in these methods. First, the time complexity would be unbearable when the number of attributes is large. Second, the assumption used to model the attribute hierarchy is too strong so that it may lose some information of the hierarchy and is not flexible enough to fit all situations. To address these limitations, we propose a novel Bayesian network-based Hierarchical Cognitive Diagnosis Framework (HierCDF), which enables many
arXiv (Cornell University), Sep 17, 2003
Methodology of educational measurement and assessment, 2017
Systems of teams with overlapping members arise in employment, training, and educational contexts... more Systems of teams with overlapping members arise in employment, training, and educational contexts. Team interdependence in these systems can confound analyses that aim to account for both individual and team attributes in studying team formation and performance. This chapter introduces bipartite networks for modeling teams with overlapping members. In these networks, individuals and teams are represented by two different types of nodes with links representing team affiliation. Two methods for analysis of bipartite networks with individual and team attributes are reviewed, exponential random graph models (ERGMs) and correspondence analysis (CA). Examples, discussions, and comparisons are provided for both methods.
Decision Support Systems
Enterprise collaboration technologies (ECTs) are increasingly recognized for supporting effective... more Enterprise collaboration technologies (ECTs) are increasingly recognized for supporting effective and efficient digital collaboration, such as decision-making activities, among employees. Given the social and collaborative nature of ECT use, social network theory offers important and helpful insights into how and why employees' social network relations facilitate their ECT use. However, existing research primarily examines the effects of a single social network relation or several social network relations separately, without applying a holistic approach to investigate the joint effect of multiple social network relations on ECT use. Drawing on a novel technique of fuzzy-set qualitative comparative analysis (fsQCA) and social network analysis, this study explores how multiple social network relations (i.e., advice, friendship, and communication) collectively influence ECT use. Using multi-source data from 178 employees in the human resources department of a global technology company, we identify several configurations of multiple social network relations associated with high ECT use and low ECT use. Our findings indicate that a single social network relation is insufficient to explain ECT use and should be considered alongside other social network relations. Overall, this study provides an integrative framework to unpack the complex and contingent effects of multiple social network relations on ECT use.
2022 8th International Conference on Big Data Computing and Communications (BigCom)
Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining
Cognitive diagnostic assessment is a fundamental task in intelligent education, which aims at qua... more Cognitive diagnostic assessment is a fundamental task in intelligent education, which aims at quantifying students' cognitive level on knowledge attributes. Since there exists learning dependency among knowledge attributes, it is crucial for cognitive diagnosis models (CDMs) to incorporate attribute hierarchy when assessing students. The attribute hierarchy is only explored by a few CDMs such as Attribute Hierarchy Method, and there are still two significant limitations in these methods. First, the time complexity would be unbearable when the number of attributes is large. Second, the assumption used to model the attribute hierarchy is too strong so that it may lose some information of the hierarchy and is not flexible enough to fit all situations. To address these limitations, we propose a novel Bayesian network-based Hierarchical Cognitive Diagnosis Framework (HierCDF), which enables many
Computer Supported Collaborative Learning, Jun 1, 2019
Innovative Assessment of Collaboration, 2017
In this introductory chapter we provide the context for this edited volume, describe the recent r... more In this introductory chapter we provide the context for this edited volume, describe the recent research interests around developing collaborative assessments around the world, and synthesize the major research results from the literature from different fields. The purpose of this edited volume was to bring together researchers from diverse disciplines—educational psychology, organizational psychology, learning sciences, assessment design, communications, human-computer interaction, computer science, engineering and applied science, psychometrics—who shared a research interest in examining learners and workers engaged in collaborative activity. This chapter concludes with an emphasis on how each chapter contributes to the research agenda around the measurement research questions, from how to define the constructs to how to model the data from collaborative interactions.
We applied machine learning-based automated text scoring techniques to provide immediate feedback... more We applied machine learning-based automated text scoring techniques to provide immediate feedback to students who are writing scientific arguments as part of their classroom activities. Based on previously collected and hand-scored student responses, the scoring models were trained and validated for scientific argumentation tasks requiring constructed responses. In so doing, the c-rater-ML engine extracted a set of feature variables from the hand-scored student responses. Empirical studies showed that most students actively interacted with the feedback by making revisions. When students made revisions, their scientific argument scores tended to increase significantly.
Social network analysis is concerned not only with social relations (Wellman 1988), but also more... more Social network analysis is concerned not only with social relations (Wellman 1988), but also more generally with attributes across pairs of social actors, which are referred to as dyadic attributes (Borgatti and Everett 1987, p. 243). These dyadic attributes range from shared affiliations to distances between cities to similarities in respondents’ answers to items on a questionnaire. While most network studies have investigated one-mode networks (Borgatti and Everett 1987), social network approaches are easily extended to two-mode data, such as the relationship between employees and work teams with which they are affiliated (Wasserman and Faust 1994). In two-mode networks, different types of nodes (e.g., employees and teams) are represented as different modes. Unlike typical affiliation networks (for a primer on affiliation networks, please refer to Wasserman and Faust 1994;
Systems of teams with overlapping members arise in employment, training, and educational contexts... more Systems of teams with overlapping members arise in employment, training, and educational contexts. Team interdependence in these systems can confound analyses that aim to account for both individual and team attributes in studying team formation and performance. This chapter introduces bipartite networks for modeling teams with overlapping members. In these networks, individuals and teams are represented by two different types of nodes with links representing team affiliation. Two methods for analysis of bipartite networks with individual and team attributes are reviewed, exponential random graph models (ERGMs) and correspondence analysis (CA). Examples, discussions, and comparisons are provided for both methods.
Psychological test and assessment modeling, 2017
Advances in technology result in evolving educational assessment design and implementation. The n... more Advances in technology result in evolving educational assessment design and implementation. The new generation assessments include innovative technology-enhanced items, such as simulations and game-like tasks that mimic an authentic learning experience. Two questions that arise along with the implementation of the technology-enhanced items are: (1) what data and their associated features may serve as meaningful measurement evidence, and (2) how to statistically and psychometrically characterize new data and reliably identify their features of interest. This paper focuses on one of the new data types, process data, which reflects students' procedure of solving a problem. A new model, a Markov-IRT model, is proposed to characterize and capture the unique features of each individual's response process during a problem-solving activity in scenario-based tasks. The structure of the model, its assumptions, the parameter space, and the estimation of the parameters are discussed in this paper. Furthermore, we illustrate the application of the Markov-IRT model, and discuss its usefulness in characterizing students' response processes using an empirical example based on a scenario-based task from the NAEP-TEL assessment. Lastly, we illustrate the identification and extraction of features of the students' response processes to be used as evidence for psychometric measurement.
Springer Proceedings in Mathematics & Statistics, 2019
There is a growing literature on the use of process data in digitally delivered assessments. In t... more There is a growing literature on the use of process data in digitally delivered assessments. In this study, we analyzed students’ essay writing processes using keystroke logs. Using four basic writing performance indicators, writers were grouped into four clusters, representing groups from fluent to struggling. The clusters differed significantly on the mean essay score, mean total time spent on task, and mean total number of words in the final submissions. Two of the four clusters were significantly different on the aforementioned three dimensions but not on typing skill. The higher scoring group even showed signs of less fluency than the lower scoring group, suggesting that task engagement and writing efforts might play an important role in generating better quality text. The four identified clusters further showed distinct sequential patterns over the course of the writing session on three process characteristics and, as well, differed on their editing behaviors during the writing process.
International Journal of Quantitative Research in Education, 2020
ETS Research Report Series, 2019
Since its 1947 founding, ETS has conducted and disseminated scientific research to support its pr... more Since its 1947 founding, ETS has conducted and disseminated scientific research to support its products and services, and to advance the measurement and education fields. In keeping with these goals, ETS is committed to making its research freely available to the professional community and to the general public. Published accounts of ETS research, including papers in the ETS Research Report series, undergo a formal peer-review process by ETS staff to ensure that they meet established scientific and professional standards. All such ETS-conducted peer reviews are in addition to any reviews that outside organizations may provide as part of their own publication processes. Peer review notwithstanding, the positions expressed in the ETS Research Report series and other published accounts of ETS research are those of the authors and not necessarily those of the Officers and Trustees of Educational Testing Service.
Communications in computer and information science, Dec 31, 2022
Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Aug 14, 2022
Cognitive diagnostic assessment is a fundamental task in intelligent education, which aims at qua... more Cognitive diagnostic assessment is a fundamental task in intelligent education, which aims at quantifying students' cognitive level on knowledge attributes. Since there exists learning dependency among knowledge attributes, it is crucial for cognitive diagnosis models (CDMs) to incorporate attribute hierarchy when assessing students. The attribute hierarchy is only explored by a few CDMs such as Attribute Hierarchy Method, and there are still two significant limitations in these methods. First, the time complexity would be unbearable when the number of attributes is large. Second, the assumption used to model the attribute hierarchy is too strong so that it may lose some information of the hierarchy and is not flexible enough to fit all situations. To address these limitations, we propose a novel Bayesian network-based Hierarchical Cognitive Diagnosis Framework (HierCDF), which enables many
arXiv (Cornell University), Sep 17, 2003
Methodology of educational measurement and assessment, 2017
Systems of teams with overlapping members arise in employment, training, and educational contexts... more Systems of teams with overlapping members arise in employment, training, and educational contexts. Team interdependence in these systems can confound analyses that aim to account for both individual and team attributes in studying team formation and performance. This chapter introduces bipartite networks for modeling teams with overlapping members. In these networks, individuals and teams are represented by two different types of nodes with links representing team affiliation. Two methods for analysis of bipartite networks with individual and team attributes are reviewed, exponential random graph models (ERGMs) and correspondence analysis (CA). Examples, discussions, and comparisons are provided for both methods.
Decision Support Systems
Enterprise collaboration technologies (ECTs) are increasingly recognized for supporting effective... more Enterprise collaboration technologies (ECTs) are increasingly recognized for supporting effective and efficient digital collaboration, such as decision-making activities, among employees. Given the social and collaborative nature of ECT use, social network theory offers important and helpful insights into how and why employees' social network relations facilitate their ECT use. However, existing research primarily examines the effects of a single social network relation or several social network relations separately, without applying a holistic approach to investigate the joint effect of multiple social network relations on ECT use. Drawing on a novel technique of fuzzy-set qualitative comparative analysis (fsQCA) and social network analysis, this study explores how multiple social network relations (i.e., advice, friendship, and communication) collectively influence ECT use. Using multi-source data from 178 employees in the human resources department of a global technology company, we identify several configurations of multiple social network relations associated with high ECT use and low ECT use. Our findings indicate that a single social network relation is insufficient to explain ECT use and should be considered alongside other social network relations. Overall, this study provides an integrative framework to unpack the complex and contingent effects of multiple social network relations on ECT use.
2022 8th International Conference on Big Data Computing and Communications (BigCom)
Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining
Cognitive diagnostic assessment is a fundamental task in intelligent education, which aims at qua... more Cognitive diagnostic assessment is a fundamental task in intelligent education, which aims at quantifying students' cognitive level on knowledge attributes. Since there exists learning dependency among knowledge attributes, it is crucial for cognitive diagnosis models (CDMs) to incorporate attribute hierarchy when assessing students. The attribute hierarchy is only explored by a few CDMs such as Attribute Hierarchy Method, and there are still two significant limitations in these methods. First, the time complexity would be unbearable when the number of attributes is large. Second, the assumption used to model the attribute hierarchy is too strong so that it may lose some information of the hierarchy and is not flexible enough to fit all situations. To address these limitations, we propose a novel Bayesian network-based Hierarchical Cognitive Diagnosis Framework (HierCDF), which enables many
Computer Supported Collaborative Learning, Jun 1, 2019
Innovative Assessment of Collaboration, 2017
In this introductory chapter we provide the context for this edited volume, describe the recent r... more In this introductory chapter we provide the context for this edited volume, describe the recent research interests around developing collaborative assessments around the world, and synthesize the major research results from the literature from different fields. The purpose of this edited volume was to bring together researchers from diverse disciplines—educational psychology, organizational psychology, learning sciences, assessment design, communications, human-computer interaction, computer science, engineering and applied science, psychometrics—who shared a research interest in examining learners and workers engaged in collaborative activity. This chapter concludes with an emphasis on how each chapter contributes to the research agenda around the measurement research questions, from how to define the constructs to how to model the data from collaborative interactions.
We applied machine learning-based automated text scoring techniques to provide immediate feedback... more We applied machine learning-based automated text scoring techniques to provide immediate feedback to students who are writing scientific arguments as part of their classroom activities. Based on previously collected and hand-scored student responses, the scoring models were trained and validated for scientific argumentation tasks requiring constructed responses. In so doing, the c-rater-ML engine extracted a set of feature variables from the hand-scored student responses. Empirical studies showed that most students actively interacted with the feedback by making revisions. When students made revisions, their scientific argument scores tended to increase significantly.
Social network analysis is concerned not only with social relations (Wellman 1988), but also more... more Social network analysis is concerned not only with social relations (Wellman 1988), but also more generally with attributes across pairs of social actors, which are referred to as dyadic attributes (Borgatti and Everett 1987, p. 243). These dyadic attributes range from shared affiliations to distances between cities to similarities in respondents’ answers to items on a questionnaire. While most network studies have investigated one-mode networks (Borgatti and Everett 1987), social network approaches are easily extended to two-mode data, such as the relationship between employees and work teams with which they are affiliated (Wasserman and Faust 1994). In two-mode networks, different types of nodes (e.g., employees and teams) are represented as different modes. Unlike typical affiliation networks (for a primer on affiliation networks, please refer to Wasserman and Faust 1994;
Systems of teams with overlapping members arise in employment, training, and educational contexts... more Systems of teams with overlapping members arise in employment, training, and educational contexts. Team interdependence in these systems can confound analyses that aim to account for both individual and team attributes in studying team formation and performance. This chapter introduces bipartite networks for modeling teams with overlapping members. In these networks, individuals and teams are represented by two different types of nodes with links representing team affiliation. Two methods for analysis of bipartite networks with individual and team attributes are reviewed, exponential random graph models (ERGMs) and correspondence analysis (CA). Examples, discussions, and comparisons are provided for both methods.
Psychological test and assessment modeling, 2017
Advances in technology result in evolving educational assessment design and implementation. The n... more Advances in technology result in evolving educational assessment design and implementation. The new generation assessments include innovative technology-enhanced items, such as simulations and game-like tasks that mimic an authentic learning experience. Two questions that arise along with the implementation of the technology-enhanced items are: (1) what data and their associated features may serve as meaningful measurement evidence, and (2) how to statistically and psychometrically characterize new data and reliably identify their features of interest. This paper focuses on one of the new data types, process data, which reflects students' procedure of solving a problem. A new model, a Markov-IRT model, is proposed to characterize and capture the unique features of each individual's response process during a problem-solving activity in scenario-based tasks. The structure of the model, its assumptions, the parameter space, and the estimation of the parameters are discussed in this paper. Furthermore, we illustrate the application of the Markov-IRT model, and discuss its usefulness in characterizing students' response processes using an empirical example based on a scenario-based task from the NAEP-TEL assessment. Lastly, we illustrate the identification and extraction of features of the students' response processes to be used as evidence for psychometric measurement.
Springer Proceedings in Mathematics & Statistics, 2019
There is a growing literature on the use of process data in digitally delivered assessments. In t... more There is a growing literature on the use of process data in digitally delivered assessments. In this study, we analyzed students’ essay writing processes using keystroke logs. Using four basic writing performance indicators, writers were grouped into four clusters, representing groups from fluent to struggling. The clusters differed significantly on the mean essay score, mean total time spent on task, and mean total number of words in the final submissions. Two of the four clusters were significantly different on the aforementioned three dimensions but not on typing skill. The higher scoring group even showed signs of less fluency than the lower scoring group, suggesting that task engagement and writing efforts might play an important role in generating better quality text. The four identified clusters further showed distinct sequential patterns over the course of the writing session on three process characteristics and, as well, differed on their editing behaviors during the writing process.
International Journal of Quantitative Research in Education, 2020
ETS Research Report Series, 2019
Since its 1947 founding, ETS has conducted and disseminated scientific research to support its pr... more Since its 1947 founding, ETS has conducted and disseminated scientific research to support its products and services, and to advance the measurement and education fields. In keeping with these goals, ETS is committed to making its research freely available to the professional community and to the general public. Published accounts of ETS research, including papers in the ETS Research Report series, undergo a formal peer-review process by ETS staff to ensure that they meet established scientific and professional standards. All such ETS-conducted peer reviews are in addition to any reviews that outside organizations may provide as part of their own publication processes. Peer review notwithstanding, the positions expressed in the ETS Research Report series and other published accounts of ETS research are those of the authors and not necessarily those of the Officers and Trustees of Educational Testing Service.