Papers › Non-Autoregressive Translation by Learning Target Categorical Codes

Non-Autoregressive Translation by Learning Target Categorical Codes

21 Mar 2021NAACL 2021 4arXiv:2103.11405archive 2025-07-28

Yu Bao, ShuJian Huang, Tong Xiao, Dongqi Wang, Xinyu Dai, Jiajun Chen

Non-autoregressive Transformer is a promising text generation model. However, current non-autoregressive models still fall behind their autoregressive counterparts in translation quality. We attribute this accuracy gap to the lack of dependency modeling among decoder inputs. In this paper, we propose CNAT, which learns implicitly categorical codes as latent variables into the non-autoregressive decoding. The interaction among these categorical codes remedies the missing dependencies and improves the model capacity. Experiment results show that our model achieves comparable or better performance in machine translation tasks, compared with several strong baselines.

Paper PDF Conference PDF Code

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

baoy-nlp/CNAT officialmentioned on GitHubpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Attribute Decoder Machine Translation Text Generation Translation

Results from the paper archive 2025-07-28

Task	Dataset	Model	Metric	Value	Rank at snapshot	Leaderboard	Report
Machine Translation	IWSLT2014 German-English	CNAT	BLEU score	31.15	#31 of 34	Archive leaderboard	report
Machine Translation	WMT2014 English-German	CNAT	BLEU score	26.6	#56 of 91	Archive leaderboard	report
Machine Translation	WMT2014 German-English	CNAT	BLEU score	30.75	#7 of 16	Archive leaderboard	report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

Absolute Position Encodings Adam Attention BPE Dense Connections Dropout Label Smoothing Layer Normalization Linear Layer Multi-Head Attention Position-Wise Feed-Forward Layer Residual Connection Softmax Transformer

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections