Methods › Natural Language Processing › Autoregressive Transformers › Levenshtein Transformer
Levenshtein Transformer
Introduced by Jiatao Gu et al. in Levenshtein Transformer
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
The Levenshtein Transformer (LevT) is a type of transformer that aims to address the lack of flexibility of previous decoding models. Notably, in previous frameworks, the length of generated sequences is either fixed or monotonically increased as the decoding proceeds. The authors argue this is incompatible with human-level intelligence where humans can revise, replace, revoke or delete any part of their generated text. Hence, LevT is proposed to bridge this gap by breaking the in-so-far standardized decoding mechanism and replacing it with two basic operations — insertion and deletion.
LevT is trained using imitation learning. The resulted model contains two policies and they are executed in an alternate manner. The authors argue that with this model decoding becomes more flexible. For example, when the decoder is given an empty token, it falls back to a normal sequence generation model. On the other hand, the decoder acts as a refinement model when the initial state is a low-quality generated sequence.
One crucial component in LevT framework is the learning algorithm. The authors leverage the characteristics of insertion and deletion — they are complementary but also adversarial. The algorithm they propose is called “dual policy learning”. The idea is that when training one policy (insertion or deletion), we use the output from its adversary at the previous iteration as input. An expert policy, on the other hand, is drawn to provide a correction signal.
Papers archive 2025-07-28
12 shown of 12, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.
-
Reinforcement Learning for Edit-Based Non-Autoregressive Neural Machine Translation 2 May 2024 · 0 repositories · arXiv:2405.01280
-
Towards Example-Based NMT with Multi-Levenshtein Transformers 13 Oct 2023 · 1 repository · arXiv:2310.08967Syntology ran 1 of 1 samples · 0 unverified
-
Integrating Translation Memories into Non-Autoregressive Machine Translation 12 Oct 2022 · 1 repository · arXiv:2210.06020
-
Levenshtein OCR 8 Sep 2022 · 2 repositories · arXiv:2209.03594
-
The JHU-Microsoft Submission for WMT21 Quality Estimation Shared Task 17 Sep 2021 · 0 repositories · arXiv:2109.08724
-
Levenshtein Training for Word-level Quality Estimation 12 Sep 2021 · 1 repository · arXiv:2109.05611
-
Using Perturbed Length-aware Positional Encoding for Non-autoregressive Neural Machine Translation 29 Jul 2021 · 0 repositories · arXiv:2107.13689
-
How Does Distilled Data Complexity Impact the Quality and Confidence of Non-Autoregressive Machine Translation? 27 May 2021 · 0 repositories · arXiv:2105.12900
-
EDITOR: an Edit-Based Transformer with Repositioning for Neural Machine Translation with Soft Lexical Constraints 13 Nov 2020 · 1 repository · arXiv:2011.06868
-
Expand and Filter: CUNI and LMU Systems for the WNGT 2020 Duolingo Shared Task 1 Jul 2020 · 0 repositories
-
Lexically Constrained Neural Machine Translation with Levenshtein Transformer 27 Apr 2020 · 1 repository · arXiv:2004.12681Syntology ran 1 of 4 samples · 3 unverified · 4 pointer-only (licence)
-
Levenshtein Transformer 27 May 2019 · 3 repositories · arXiv:1905.11006
Tasks archive 2025-07-28
17 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections