{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/filling-the-gaps-in-ancient-akkadian-texts-a","title":"Filling the Gaps in Ancient Akkadian Texts: A Masked Language Modelling Approach","arxiv_id":"2109.04513","date":"2021-09-09","proceeding":"EMNLP 2021 11","authors":["Koren Lazar","Benny Saret","Asaf Yehudai","Wayne Horowitz","Nathan Wasserman","Gabriel Stanovsky"],"abstract":"We present models which complete missing text given transliterations of ancient Mesopotamian documents, originally written on cuneiform clay tablets (2500 BCE - 100 CE). Due to the tablets' deterioration, scholars often rely on contextual cues to manually fill in missing parts in the text in a subjective and time-consuming process. We identify that this challenge can be formulated as a masked language modelling task, used mostly as a pretraining objective for contextualized language models. Following, we develop several architectures focusing on the Akkadian language, the lingua franca of the time. We find that despite data scarcity (1M tokens) we can achieve state of the art performance on missing tokens prediction (89% hit@5) using a greedy decoding scheme and pretraining on data from other languages and different time periods. Finally, we conduct human evaluations showing the applicability of our models in assisting experts to transcribe texts in extinct languages.","url_abs":"https://arxiv.org/abs/2109.04513v2","url_pdf":"https://arxiv.org/pdf/2109.04513v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"filling-the-gaps-in-ancient-akkadian-texts-a","repo_url":"https://github.com/slab-nlp/akk","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"language-modelling","task_name":"Language Modelling"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2109.04513","atlas_url":"https://app.syntology.ai/?focus=2109.04513","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2109.04513"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/SLAB-NLP/Akk","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":11},"by_repo_kind":{"official":{"samples":11,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"dc0f42152bd71f09","entry":"calc_wind_around_ind","repo":"SLAB-NLP/Akk","repo_kind":"official","path":"akkadian_bert/utils.py","file_url":"https://github.com/SLAB-NLP/Akk/blob/HEAD/akkadian_bert/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dc0f42152bd71f09"}},{"code_sha256_prefix":"85176e184025c2e0","entry":"get_bert_path","repo":"SLAB-NLP/Akk","repo_kind":"official","path":"akkadian_bert/train_bert.py","file_url":"https://github.com/SLAB-NLP/Akk/blob/HEAD/akkadian_bert/train_bert.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"85176e184025c2e0"}},{"code_sha256_prefix":"724cd694c6e42708","entry":"get_catalog","repo":"SLAB-NLP/Akk","repo_kind":"official","path":"RAW/get_langs.py","file_url":"https://github.com/SLAB-NLP/Akk/blob/HEAD/RAW/get_langs.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"724cd694c6e42708"}},{"code_sha256_prefix":"931f7b05fb434414","entry":"get_concatenated_files","repo":"SLAB-NLP/Akk","repo_kind":"official","path":"akkadian_bert/utils.py","file_url":"https://github.com/SLAB-NLP/Akk/blob/HEAD/akkadian_bert/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"931f7b05fb434414"}},{"code_sha256_prefix":"26fa0b6bedc5dd1e","entry":"get_enc_labels","repo":"SLAB-NLP/Akk","repo_kind":"official","path":"akkadian_bert/datasets_bert.py","file_url":"https://github.com/SLAB-NLP/Akk/blob/HEAD/akkadian_bert/datasets_bert.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"26fa0b6bedc5dd1e"}},{"code_sha256_prefix":"4f75983bc1a014c4","entry":"get_files","repo":"SLAB-NLP/Akk","repo_kind":"official","path":"RAW/work_on_files.py","file_url":"https://github.com/SLAB-NLP/Akk/blob/HEAD/RAW/work_on_files.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4f75983bc1a014c4"}},{"code_sha256_prefix":"983efcbf97af1179","entry":"get_language","repo":"SLAB-NLP/Akk","repo_kind":"official","path":"RAW/get_langs.py","file_url":"https://github.com/SLAB-NLP/Akk/blob/HEAD/RAW/get_langs.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"983efcbf97af1179"}},{"code_sha256_prefix":"fb4bc8bf9816564c","entry":"get_period","repo":"SLAB-NLP/Akk","repo_kind":"official","path":"RAW/get_langs.py","file_url":"https://github.com/SLAB-NLP/Akk/blob/HEAD/RAW/get_langs.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fb4bc8bf9816564c"}},{"code_sha256_prefix":"5b9c23c30e55b67e","entry":"natural_number","repo":"SLAB-NLP/Akk","repo_kind":"official","path":"akkadian_bert/utils.py","file_url":"https://github.com/SLAB-NLP/Akk/blob/HEAD/akkadian_bert/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5b9c23c30e55b67e"}},{"code_sha256_prefix":"6b76badec7a176c5","entry":"read_file","repo":"SLAB-NLP/Akk","repo_kind":"official","path":"RAW/work_on_files.py","file_url":"https://github.com/SLAB-NLP/Akk/blob/HEAD/RAW/work_on_files.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6b76badec7a176c5"}},{"code_sha256_prefix":"621e7c8c7d006787","entry":"read_project","repo":"SLAB-NLP/Akk","repo_kind":"official","path":"RAW/work_on_files.py","file_url":"https://github.com/SLAB-NLP/Akk/blob/HEAD/RAW/work_on_files.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"621e7c8c7d006787"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}