Papers › CAE: Mechanism to Diminish the Class Imbalanced in SLU Slot Filling Task

CAE: Mechanism to Diminish the Class Imbalanced in SLU Slot Filling Task

21 Sep 2022Advances in Computational Collective Intelligence 2022 9archive 2025-07-28

Nguyen Minh Phuong, Tung Le, Nguyen Le Minh

Spoken Language Understanding (SLU) task is a wide application task in Natural Language Processing. In the success of the pre-trained BERT model, NLU is addressed by Intent Classification and Slot Filling task with significant improvement performance. However, classed imbalance problem in NLU has not been carefully investigated, while this problem in Semantic Parsing datasets is frequent. Therefore, this work focuses on diminishing this problem. We proposed a BERT-based architecture named JointBERT Classify Anonymous Entity (JointBERT-CAE) that improves the performance of the system on three Semantic Parsing datasets ATIS, Snips, ATIS Vietnamese, and a well-known Named Entity Recognize (NER) dataset CoNLL2003. In JointBERT-CAE architecture, we use multitask joint-learning to split conventional Slot Filling task into two sub-task, detect Anonymous Entity by Sequence tagging and Classify recognized anonymous entities tasks. The experimental results show the solid improvement of JointBERT-CAE when compared with BERT on all datasets, as well as the wide applicable capacity to other NLP tasks using the Sequence Tagging technique.

Paper PDF Code

Code

phuongnm94/JointBERT_CAE pytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Intent Classification Intent Classification and Slot Filling Intent Detection NER Semantic Parsing Slot Filling Spoken Language Understanding intent-classification slot-filling

Results from the paper archive 2025-07-28

Task	Dataset	Model	Metric	Value	Rank at snapshot	Leaderboard	Report
Intent Detection	ATIS	JointBERT-CAE	Accuracy	97.5	#11 of 16	Archive leaderboard	report
Intent Detection	ATIS (vi)	JointBERT-CAE	Intent Accuracy	97.7	#1 of 1	Archive leaderboard	report
Intent Detection	SNIPS	JointBERT-CAE	Accuracy	98.3	#3 of 10	Archive leaderboard	report
Slot Filling	ATIS	JointBERT-CAE	F1	0.961	#4 of 14	Archive leaderboard	report
Slot Filling	ATIS (vi)	JointBERT-CAE	Slot F1	95.5	#1 of 1	Archive leaderboard	report
Slot Filling	SNIPS	JointBERT-CAE	F1	97.0	#2 of 10	Archive leaderboard	report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

Adam Attention Attention Dropout BERT Dense Connections Dropout Layer Normalization Linear Layer Linear Warmup With Linear Decay Multi-Head Attention Residual Connection Softmax Weight Decay WordPiece

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections