Papers › Improving Rare Word Translation With Dictionaries and Attention Masking

Improving Rare Word Translation With Dictionaries and Attention Masking

17 Aug 2024arXiv:2408.09075archive 2025-07-28

Kenneth J. Sible, David Chiang

In machine translation, rare words continue to be a problem for the dominant encoder-decoder architecture, especially in low-resource and out-of-domain translation settings. Human translators solve this problem with monolingual or bilingual dictionaries. In this paper, we propose appending definitions from a bilingual dictionary to source sentences and using attention masking to link together rare words with their definitions. We find that including definitions for rare words improves performance by up to 1.0 BLEU and 1.6 MacroF1.

PaperPDFCode

Code

kennethsible/dictionary-attention officialmentioned in paperpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

DecoderMachine TranslationTranslationWord Translation

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Methods

AttentionSoftmax

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections