{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/word-alignment-by-fine-tuning-embeddings-on","title":"Word Alignment by Fine-tuning Embeddings on Parallel Corpora","arxiv_id":"2101.08231","date":"2021-01-20","proceeding":"EACL 2021 2","authors":["Zi-Yi Dou","Graham Neubig"],"abstract":"Word alignment over parallel corpora has a wide variety of applications, including learning translation lexicons, cross-lingual transfer of language processing tools, and automatic evaluation or analysis of translation outputs. The great majority of past work on word alignment has worked by performing unsupervised learning on parallel texts. Recently, however, other work has demonstrated that pre-trained contextualized word embeddings derived from multilingually trained language models (LMs) prove an attractive alternative, achieving competitive results on the word alignment task even in the absence of explicit training on parallel data. In this paper, we examine methods to marry the two approaches: leveraging pre-trained LMs but fine-tuning them on parallel text with objectives designed to improve alignment quality, and proposing methods to effectively extract alignments from these fine-tuned models. We perform experiments on five language pairs and demonstrate that our model can consistently outperform previous state-of-the-art models of all varieties. In addition, we demonstrate that we are able to train multilingual word aligners that can obtain robust performance on different language pairs. Our aligner, AWESOME (Aligning Word Embedding Spaces of Multilingual Encoders), with pre-trained models is available at https://github.com/neulab/awesome-align","url_abs":"https://arxiv.org/abs/2101.08231v4","url_pdf":"https://arxiv.org/pdf/2101.08231v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"word-alignment-by-fine-tuning-embeddings-on","repo_url":"https://github.com/neulab/awesome-align","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"word-alignment-by-fine-tuning-embeddings-on","repo_url":"https://github.com/BramVanroy/astred","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"word-alignment-by-fine-tuning-embeddings-on","repo_url":"https://github.com/ikergarcia1996/easy-label-projection","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"cross-lingual-transfer","task_name":"Cross-Lingual Transfer"},{"task_slug":"translation","task_name":"Translation"},{"task_slug":"word-alignment","task_name":"Word Alignment"},{"task_slug":"word-embeddings","task_name":"Word Embeddings"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2101.08231","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2101.08231"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ikergarcia1996/easy-label-projection","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/neulab/awesome-align","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/BramVanroy/astred","reach":{"status":"unanswered"}}],"summary":{"ran_honours":1,"ran_draft_wrong":1},"by_repo_kind":{"official":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"4c0ba2c83abf3771","entry":"find_offsets","repo":"neulab/awesome-align","repo_kind":"official","path":"awesome_align/run_align.py","file_url":"https://github.com/neulab/awesome-align/blob/HEAD/awesome_align/run_align.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"4c0ba2c83abf3771"}},{"code_sha256_prefix":"59e42c11f6fcca4b","entry":"open_writer_list","repo":"neulab/awesome-align","repo_kind":"official","path":"awesome_align/run_align.py","file_url":"https://github.com/neulab/awesome-align/blob/HEAD/awesome_align/run_align.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"59e42c11f6fcca4b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}