Ensembling Predictions of Student Knowledge within Intelligent Tutoring Systems (original) (raw)
Related papers
The Sum is Greater than the Parts : Ensembling Student Knowledge Models in ASSISTments
2011
Recent research has had inconsistent results as to the utility of ensembling different approaches towards modeling student knowledge and skill within interactive learning environments. While work in the 2010 KDD Cup data set has shown benefits from ensembling, work in the Genetics Tutor has failed to show benefits. We hypothesize that the key factor has been data set size. We explore the potential for ensembling in a data set drawn from a different tutoring system, The ASSISTments Platform, which contains 15 times the number of responses of the Genetics Tutor data set. Within this data set, ensemble approaches were more effective than any single method with the best ensemble approach producing predictions of student performance 10% better than the best individual student knowledge model.
Clustering students to generate an ensemble to improve standard test score predictions
2011
In typical assessment student are not given feedback, as it is harder to predict student knowledge if it is changing during testing. Intelligent Tutoring systems, that offer assistance while the student is participating, offer a clear benefit of assisting students, but how well can they assess students? What is the trade off in terms of assessment accuracy if we allow student to be assisted on an exam. In a prior study, we showed the assistance with assessments quality to be equal. In this work, we introduce a more sophisticated method by which we can ensemble together multiple models based upon clustering students. We show that in fact, the assessment quality as determined by the assistance data is a better estimator of student knowledge. The implications of this study suggest that by using computer tutors for assessment, we can save much instructional time that is currently used for just assessment.
Predicting Students Performance with SimStudent that Learns Cognitive Skills from Observation
2000
SimStudent is a machine-learning agent that learns cognitive skills by demonstration. SimStudent was originally built as a building block for Cognitive Tutor Authoring Tools to help an author build a cognitive model without heavy programming. In this paper, we evaluate a second use of SimStudent for student modeling for Intelligent Tutoring Systems. The basic idea is to have SimStudent observe
The sum is greater than the parts: ensembling models of student knowledge in educational software
Sigkdd Explorations, 2012
Many competing models have been proposed in the past decade for predicting student knowledge within educational software. Recent research attempted to combine these models in an effort to improve performance but have yielded inconsistent results. While work in the 2010 KDD Cup data set showed the benefits of ensemble methods, work in the Genetics Tutor failed to show similar benefits.
Predicting Students' Performance with SimStudent: Learning Cognitive Skills from Observation
2007
SimStudent is a machine-learning agent that learns cognitive skills by demonstration. SimStudent was originally built as a building block for Cognitive Tutor Authoring Tools to help an author build a cognitive model without significant programming. In this paper, we evaluate a second use of SimStudent, viz., student modeling for Intelligent Tutoring Systems. The basic idea is to have SimStudent observe human students solving problems. It then creates a cognitive model that can replicate the students' performance. If the model is accurate, it would predict the human students' performance on novel problems. An evaluation study showed that when trained on 15 problems, SimStudent accurately predicted the human students' correct behavior on the novel problems more than 80% of the time. However, the current implementation of SimStudent does not accurately predict when the human students make errors.
A Bayes Net Toolkit for Student Modeling in Intelligent Tutoring Systems
Lecture Notes in Computer Science, 2006
This paper describes an effort to model a student's changing knowledge state during skill acquisition. Dynamic Bayes Nets (DBNs) provide a powerful way to represent and reason about uncertainty in time series data, and are therefore well-suited to model student knowledge. Many general-purpose Bayes net packages have been implemented and distributed; however, constructing DBNs often involves complicated coding effort. To address this problem, we introduce a tool called BNT-SM. BNT-SM inputs a data set and a compact XML specification of a Bayes net model hypothesized by a researcher to describe causal relationships among student knowledge and observed behavior. BNT-SM generates and executes the code to train and test the model using the Bayes Net Toolbox [1]. Compared to the BNT code it outputs, BNT-SM reduces the number of lines of code required to use a DBN by a factor of 5. In addition to supporting more flexible models, we illustrate how to use BNT-SM to simulate Knowledge Tracing (KT) , an established technique for student modeling. The trained DBN does a better job of modeling and predicting student performance than the original KT code (Area Under Curve = 0.610 > 0.568), due to differences in how it estimates parameters.
The role of probability-based inference in an intelligent tutoring system
User Modeling and User-Adapted Interaction, 1996
Pursuit of efficient probability-based inference in complex networks of interdependent variables is an active topic in current statistical research, spurred by such diverse applications as forecasting, pedigree analysis, troubleshooting, and medical diagnosis. This paper concerns the potential role of Bayesian inference networks for updating student models in intelligent tutoring systems (ITSs). Basic concepts and tools of the approach are reviewed, but emphasis is on special considerations that arise in the ITS context. We explore how this approach can support generalized claims about aspects of student proficiency through the combination of detailed epistemic analysis of particular actions within a system with probability-based inference. The psychology of learning in the domain and the instructional approach are seen to play crucial roles. Ideas are illustrated with HYDRIVE, an ITS for learning to troubleshoot an aircraft hydraulics system.
Toward Near Zero-Parameter Prediction Using a Computational Model of Student Learning
2019
Computational models of learning can be powerful tools to test educational technologies, automate the authoring of instructional software, and advance theories of learning. These mechanistic models of learning, which instantiate computational theories of the learning process, are capable of making predictions about learners’ performance in instructional technologies given only the technology itself without fitting any parameters to existing learners’ data. While these so call “zero-parameter” models have been successful in modeling student learning in intelligent tutoring systems they still show systematic deviation from human learning performance. One deviation stems from the computational models’ lack of prior knowledge—all models start off as a blank slate—leading to substantial differences in performance at the first practice opportunity. In this paper, we explore three different strategies for accounting for prior knowledge within computational models of learning and the effect...
Lecture Notes in Computer Science, 2006
The ASSISTment system was used by over 600 students in 2004-05 school year as part of their math class. While in [7] we reported student learning within the ASSISTment system, in this paper we focus on the assessment aspect. Our approach is to use data that the system collected through a year to tracking student learning and thus estimate their performance on a high-stake state test (MCAS) at the end of the year. Because our system is an intelligent tutoring system, we are able to log how much assistance students needed to solve problems (how many hints students requested and how many attempts they had to make). In this paper, our goal is to determine if the models we built by taking the assistance information into account could predict students' test scores better. We present some positive evidence that shows our goal is achieved.
A Qualitative Comparison of Techniques for Student Modelling in Intelligent Tutoring Systems
nternational journal of advanced research in computer and communication engineering, 2020
Wise Tutoring Systems (ITS) are intuitive learning conditions dependent on guidance helped by P.C.s. The insight of these frameworks is, to a great extent, ascribed to their capacity to adjust to a particular understudy during the educating cycle. As a rule, the variation cycle depicts by three stages: (I) getting the data about the understudy, (ii) preparing the data to introduce and refresh an understudy model, also, (iii) utilizing the understudy model to give the transformation. In this paper, we considered viewpoints related to understudy displaying (S.M.) in Intelligent Tutoring Systems. First, we make a subjective examination of two procedures: Bayesian Networks (B.N.) and Case-based Reasoning (CBR) for S.M. We apply the two strategies to a contextual analysis concerning the advancement of an ITS for e-learning in the clinical space. At last, we talk about the outcomes acquired.