Papers › Self-Explaining Structures Improve NLP Models
Self-Explaining Structures Improve NLP Models
Zijun Sun, Chun Fan, Qinghong Han, Xiaofei Sun, Yuxian Meng, Fei Wu, Jiwei Li
Existing approaches to explaining deep learning models in NLP usually suffer from two major drawbacks: (1) the main model and the explaining model are decoupled: an additional probing or surrogate model is used to interpret an existing model, and thus existing explaining tools are not self-explainable; (2) the probing model is only able to explain a model's predictions by operating on low-level features by computing saliency scores for individual words but are clumsy at high-level text units such as phrases, sentences, or paragraphs. To deal with these two issues, in this paper, we propose a simple yet general and effective self-explaining framework for deep learning models in NLP. The key point of the proposed framework is to put an additional layer, as is called by the interpretation layer, on top of any existing NLP model. This layer aggregates the information for each text span, which is then associated with a specific weight, and their weighted combination is fed to the softmax function for the final prediction. The proposed model comes with the following merits: (1) span weights make the model self-explainable and do not require an additional probing model for interpretation; (2) the proposed model is general and can be adapted to any existing deep learning structures in NLP; (3) the weight associated with each text span provides direct importance scores for higher-level text units such as phrases and sentences. We for the first time show that interpretability does not come at the cost of performance: a neural model of self-explaining features obtains better performances than its counterpart without the self-explaining nature, achieving a new SOTA performance of 59.1 on SST-5 and a new SOTA performance of 92.3 on SNLI.
In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Natural Language Inference | SNLI | RoBERTa-large+Self-Explaining | % Test Accuracy | 92.3 | #5 of 98 | Archive leaderboard | report |
| Natural Language Inference | SNLI | RoBERTa-large+Self-Explaining | Parameters | 340 | #5 of 98 | Archive leaderboard | report |
| Natural Language Inference | SNLI | RoBERTa-large + self-explaining layer | % Test Accuracy | 92.3 | #6 of 98 | Archive leaderboard | report |
| Natural Language Inference | SNLI | RoBERTa-large + self-explaining layer | % Train Accuracy | ? | #6 of 98 | Archive leaderboard | report |
| Natural Language Inference | SNLI | RoBERTa-large + self-explaining layer | Parameters | 355m+ | #6 of 98 | Archive leaderboard | report |
| Paraphrase Identification | Quora Question Pairs | 1-3[0.8pt/2pt] Random | Accuracy | 80 | #27 of 31 | Archive leaderboard | report |
| Sentiment Analysis | SST-5 Fine-grained classification | RoBERTa-large+Self-Explaining | Accuracy | 59.1 | #3 of 31 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Methods
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections