Papers › Towards Robust Knowledge Tracing Models via k-Sparse Attention

Towards Robust Knowledge Tracing Models via k-Sparse Attention

24 Jul 2024arXiv:2407.17097archive 2025-07-28

Shuyan Huang, Zitao Liu, Xiangyu Zhao, Weiqi Luo, Jian Weng

Knowledge tracing (KT) is the problem of predicting students' future performance based on their historical interaction sequences. With the advanced capability of capturing contextual long-term dependency, attention mechanism becomes one of the essential components in many deep learning based KT (DLKT) models. In spite of the impressive performance achieved by these attentional DLKT models, many of them are often vulnerable to run the risk of overfitting, especially on small-scale educational datasets. Therefore, in this paper, we propose sparseKT, a simple yet effective framework to improve the robustness and generalization of the attention based DLKT approaches. Specifically, we incorporate a k-selection module to only pick items with the highest attention scores. We propose two sparsification heuristics : (1) soft-thresholding sparse attention and (2) top-K sparse attention. We show that our sparseKT is able to help attentional KT models get rid of irrelevant student interactions and have comparable predictive performance when compared to 11 state-of-the-art KT models on three publicly available real-world educational datasets. To encourage reproducible research, we make our data and code publicly available at https://github.com/pykt-team/pykt-toolkit (We merged our model to the pyKT benchmark at https://pykt.org/.).

PaperPDFCode

Syntology This paper's arXiv id is not in Syntology's Atlas id list (549,449 ids, as this build read them; the read time is in the build record), so this page links to no Syntology page for it. The list is not the whole graph; Syntology's feed shows papers as they enter the graph.

Code

pykt-team/pykt-toolkit officialmentioned in paperpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to simpleKT: A Simple But Tough-to-Beat Baseline for Knowledge Tracing (arXiv:2302.06881; that paper's own run record: 10 ran (of which 5 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 0 violated, 6 with no contract checked; 4 where Syntology's instrument failed) · 8 unverified) and Improving Interpretability of Deep Sequential Knowledge Tracing Models with... (arXiv:2302.06885; that paper's own run record: 8 ran (of which 5 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 0 violated, 6 with no contract checked; 2 where Syntology's instrument failed) · 3 unverified); nothing checks that they implement this paper's method. report
ZhijieXiong/pyedmine pytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Deep Knowledge Tracing (arXiv:1506.05908; that paper's own run record: 10 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 0 honoured, 1 violated, 9 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified), simpleKT: A Simple But Tough-to-Beat Baseline for Knowledge Tracing (arXiv:2302.06881; that paper's own run record: 10 ran (of which 5 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 0 violated, 6 with no contract checked; 4 where Syntology's instrument failed) · 8 unverified) and Improving Interpretability of Deep Sequential Knowledge Tracing Models with... (arXiv:2302.06885; that paper's own run record: 8 ran (of which 5 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 0 violated, 6 with no contract checked; 2 where Syntology's instrument failed) · 3 unverified); nothing checks that they implement this paper's method. report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

A paper named beside a repository with no sample linked to this paper is shown with that paper's own run record, not this paper's: “ran” means executed on a synthesized input, not that the code is correct, and “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. Papers are listed in arXiv-id order, at most three per repository.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Knowledge Tracing

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Methods

AttentionSoftmax

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections