Properties of the Bayesian Knowledge Tracing Model (original) (raw)

Hidden Markov IRT model as a generalization of Bayesian Knowledge Tracing

2019

To develop learner's ability, a teacher should grasp the learner's knowledge state accurately in a learning process.For this purpose, Bayesian Knowledge Tracing (BKT) has been proposed to infer learner's knowledge state.Although conventional BKT models learner's knowledge state as a discrete value, the learner's knowledge state must be contentious. Based on this idea, we propose a Hidden Markov IRT model as a generalization of Bayesian Knowledge Tracing. In the proposed model, learner's knowledge state takes a continuous value and change according to a Hidden Markov process in a learning process. The proposed model estimates the optimal value of the degree of learner's mastering knowledge from learning data.From some numerical experiments, we demonstrate that the proposed model improves the estimation accuracy of the learner's knowledge state.

Comparing Bayesian Knowledge Tracing Model Against Naïve Mastery Model

Springer eBooks, 2021

We conducted a study to see if using Bayesian Knowledge Tracing (BKT) models would save time and problems in programming tutors. We used legacy data collected by two programming tutors to compute BKT models for every concept covered by each tutor. The novelty of our model was that slip and guess parameters were computed for every problem presented by each tutor. Next, we used cross-validation to evaluate whether the resulting BKT model would have reduced the number of practice problems solved and time spent by the students represented in the legacy data. We found that in 64.23% of the concepts, students would have saved time with the BKT model. The savings varied among concepts. Overall, students would have saved a mean of 1.28 min and 1.23 problems per concept. We also found that BKT models were more effective at saving time and problems on harder concepts.

Evaluating Bayesian Knowledge Tracing for Estimating Learner Proficiency and Guiding Learner Behavior

Proceedings of the Seventh ACM Conference on Learning @ Scale, 2020

Open navigation online learning systems allow learners to choose the next learning activity. These systems can be instrumented to provide learners with feedback to help them choose the next learning activity. One type of feedback is providing an estimate of the learner's current skill proficiency. A learner can then choose to skip the remaining learning activities for that skill after achieving proficiency in that skill. In this paper, we investigate whether predicting proficiency and communicating it to learners can save time for learners within a course. We evaluate the accuracy of the BKT based proficiency prediction framework for learner's proficiency prediction which considers one attempt per question. We extend the proficiency prediction framework to include multiple attempts at individual questions and show that it is more accurate in proficiency prediction than BKT based proficiency prediction framework. We discuss the potential implications of attempt enhanced framework on the learners' behavior for open navigation online learning systems.

Individualizing Bayesian Knowledge Tracing. Are Skill Parameters More Important Than Student Parameters?

2016

Bayesian Knowledge Tracing (BKT) models were in active use in the Intelligent Tutoring Systems (ITS) field for over 20 years. They have been intensively studied, and a number of useful extensions to them were proposed and experimentally tested. Among the most widely researched extensions to BKT models are various types of individualization. Individualization, broadly defined, is a way to account for variability in students that are working with the ITS that uses BKT model to represent and track student learning. One of the approaches to individualizing BKT is to split its parameters into per-skill and per-student components. In this work, we are proposing an approach to individualizing BKT that is based on Hierarchical Bayesian Models (HBM) and, in addition to capturing student-level variability in the data, weighs the contribution of per-student and per-skill effects to the overall variance in the data.

Tracking student progress in a game-like learning environment with a Monte Carlo Bayesian knowledge tracing model

Proceedings of the Fifth International Conference on Learning Analytics And Knowledge, 2015

The Bayesian Knowledge Tracing (BKT) model is a popular model used for tracking student progress in learning systems such as an intelligent tutoring system. However, the model is not free of problems. Well-recognized problems include the identifiability problem and the empirical degeneracy problem. Unfortunately, these problems are still poorly understood and how they should be dealt with in practice is unclear. Here, we analyze the mathematical structure of the BKT model, identify a source of the difficulty, and construct a simple Monte Carlo BKT model to analyze the problem in real data. Using the student activity data obtained from the ramp task module at the Concord Consortium, we find that the Monte Carlo BKT analysis is capable of detecting the identifiability problem and the empirical degeneracy problem, and, more generally, gives an excellent summary of the student learning data. In particular, the student activity monitoring parameter M emerges as the central parameter.

Individualized Bayesian Knowledge Tracing Models

Lecture Notes in Computer Science, 2013

Bayesian Knowledge Tracing (BKT)[1] is a user modeling method extensively used in the area of Intelligent Tutoring Systems. In the standard BKT implementation, there are only skill-specific parameters. However, a large body of research strongly suggests that studentspecific variability in the data, when accounted for, could enhance model accuracy . In this work, we revisit the problem of introducing student-specific parameters into BKT on a larger scale. We show that student-specific parameters lead to a tangible improvement when predicting the data of unseen students, and that parameterizing students' speed of learning is more beneficial than parameterizing a priori knowledge.

Spectral Bayesian Knowledge Tracing

2015

Bayesian Knowledge Tracing (BKT) has been in wide use for modeling student skill acquisition in Intelligent Tutoring Systems (ITS). BKT tracks and updates student’s latent mastery of a skill as a probability distribution of a binary variable. BKT does so by accounting for observed student successes in applying the skill correctly, where success is also treated as a binary variable. While the BKT served the ITS community well, representing both the latent state and the observed performance as binary variables is, nevertheless, a simplification. In addition, BKT as a two-state and two-observation first-order HMM is prone to noise in the data. In this paper, we present work that uses feature compensation and model compensation paradigms in an attempt to conceptualize a more flexible and robust BKT model. Validation of this approach on the KDD Cup 2010 data shows a tangible boost in model accuracy well over the improvements reported in the literature.

Direct Estimation of the Minimum RSS Value for Training Bayesian Knowledge Tracing Parameters

2015

Student modeling can help guide the behavior of a cognitive tutor system and provide insight to researchers on understanding how students learn. In this context, Bayesian Knowledge Tracing (BKT) is one of the most popular knowledge inference models due to its predictive accuracy, interpretability and ability to infer student knowledge. However, the most popular methods for training the parameters of BKT have some problems, such as identifiability, local minima, degenerate parameters and computational cost during fitting. In this paper we address some of the issues of one of these training models, BKT Brute Force. Instead of finding the parameter values that provide the lowest Residual Sum of Squares (RSS), we estimate this minimum RSS value from some a priori known values of the skill. From there we perform some preliminary analysis to improve our knowledge of the relationship between the RSS, from BKT-BF, and the four BKT parameters.

An Introduction to Bayesian Knowledge Tracing with pyBKT

Psych

This study aims to introduce Bayesian Knowledge Tracing (BKT), a probabilistic model used in educational data mining to estimate learners’ knowledge states over time. It also provides a practical guide to estimating BKT models using the pyBKT library available in Python. The first section presents an overview of BKT by explaining its theoretical foundations and advantages in modeling individual learning processes. In the second section, we describe different variants of the standard BKT model based on item response theory (IRT). Next, we demonstrate the estimation of BKT with the pyBKT library in Python, outlining data pre-processing steps, parameter estimation, and model evaluation. Different cases of knowledge tracing tasks illustrate how BKT estimates learners’ knowledge states and evaluates prediction accuracy. The results highlight the utility of BKT in capturing learners’ knowledge states dynamically. We also show that the model parameters of BKT resemble the parameters from l...

Navigating the parameter space of Bayesian Knowledge Tracing models: Visualizations of the convergence of the Expectation Maximization algorithm

2010

Bayesian Knowledge Tracing (KT) models are employed by the cognitive tutors in order to determine student knowledge based on four parameters: learn rate, prior, guess and slip. A commonly used algorithm for learning these parameter values from data is the Expectation Maximization (EM) algorithm. Past work, however, has suggested that with four free parameters the standard KT model is prone to converging to erroneous degenerate states depending on the initial values of these four parameters. In this work we simulate data from a model with known parameter values and then run a grid search over the parameter initialization space of KT to map out which initial values lead to erroneous learned parameters. Through analysis of convergence and error surface visualizations we found that the initial parameter values leading to a degenerate state are not scattered randomly throughput the parameter space but instead exist on a surface with predictable boundaries. A recently introduced extension...