{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/olmo-accelerating-the-science-of-language","title":"OLMo: Accelerating the Science of Language Models","arxiv_id":"2402.00838","date":"2024-02-01","proceeding":null,"authors":["Dirk Groeneveld","Iz Beltagy","Pete Walsh","Akshita Bhagia","Rodney Kinney","Oyvind Tafjord","Ananya Harsh Jha","Hamish Ivison","Ian Magnusson","Yizhong Wang","Shane Arora","David Atkinson","Russell Authur","Khyathi Raghavi Chandu","Arman Cohan","Jennifer Dumas","Yanai Elazar","Yuling Gu","Jack Hessel","Tushar Khot","William Merrill","Jacob Morrison","Niklas Muennighoff","Aakanksha Naik","Crystal Nam","Matthew E. Peters","Valentina Pyatkin","Abhilasha Ravichander","Dustin Schwenk","Saurabh Shah","Will Smith","Emma Strubell","Nishant Subramani","Mitchell Wortsman","Pradeep Dasigi","Nathan Lambert","Kyle Richardson","Luke Zettlemoyer","Jesse Dodge","Kyle Lo","Luca Soldaini","Noah A. Smith","Hannaneh Hajishirzi"],"abstract":"Language models (LMs) have become ubiquitous in both NLP research and in commercial product offerings. As their commercial importance has surged, the most powerful models have become closed off, gated behind proprietary interfaces, with important details of their training data, architectures, and development undisclosed. Given the importance of these details in scientifically studying these models, including their biases and potential risks, we believe it is essential for the research community to have access to powerful, truly open LMs. To this end, we have built OLMo, a competitive, truly Open Language Model, to enable the scientific study of language models. Unlike most prior efforts that have only released model weights and inference code, we release OLMo alongside open training data and training and evaluation code. We hope this release will empower the open research community and inspire a new wave of innovation.","url_abs":"https://arxiv.org/abs/2402.00838v4","url_pdf":"https://arxiv.org/pdf/2402.00838v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"olmo-accelerating-the-science-of-language","repo_url":"https://github.com/allenai/olmo","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"olmo-accelerating-the-science-of-language","repo_url":"https://github.com/ningyuxu/tip_of_tongue","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"olmo-accelerating-the-science-of-language","repo_url":"https://github.com/yangyucheng000/University/tree/main/model-1/olmo","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null}],"tasks":[{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2402.00838","atlas_url":"https://app.syntology.ai/?focus=2402.00838","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.00838"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ningyuxu/tip_of_tongue","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/allenai/olmo","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yangyucheng000/University/tree/main/model-1/olmo","reach":null}],"summary":{"ran":6,"unverified":7},"by_repo_kind":{"listed":{"samples":13,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"c445b3401a0c2dc5","entry":"binarize","repo":"ningyuxu/tip_of_tongue","repo_kind":"listed","path":"calf/utils/file.py","file_url":"https://github.com/ningyuxu/tip_of_tongue/blob/HEAD/calf/utils/file.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c445b3401a0c2dc5"}},{"code_sha256_prefix":"1b804ad33840c905","entry":"extract","repo":"ningyuxu/tip_of_tongue","repo_kind":"listed","path":"calf/utils/file.py","file_url":"https://github.com/ningyuxu/tip_of_tongue/blob/HEAD/calf/utils/file.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1b804ad33840c905"}},{"code_sha256_prefix":"1b6c470bcbe26474","entry":"isfullwidth","repo":"ningyuxu/tip_of_tongue","repo_kind":"listed","path":"calf/utils/common.py","file_url":"https://github.com/ningyuxu/tip_of_tongue/blob/HEAD/calf/utils/common.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1b6c470bcbe26474"}},{"code_sha256_prefix":"859c276ee0839129","entry":"islatin","repo":"ningyuxu/tip_of_tongue","repo_kind":"listed","path":"calf/utils/common.py","file_url":"https://github.com/ningyuxu/tip_of_tongue/blob/HEAD/calf/utils/common.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"859c276ee0839129"}},{"code_sha256_prefix":"ef529d3102c72be4","entry":"ispunct","repo":"ningyuxu/tip_of_tongue","repo_kind":"listed","path":"calf/utils/common.py","file_url":"https://github.com/ningyuxu/tip_of_tongue/blob/HEAD/calf/utils/common.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ef529d3102c72be4"}},{"code_sha256_prefix":"8a3a4a8cc5bb4508","entry":"wait","repo":"ningyuxu/tip_of_tongue","repo_kind":"listed","path":"calf/utils/distributed.py","file_url":"https://github.com/ningyuxu/tip_of_tongue/blob/HEAD/calf/utils/distributed.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8a3a4a8cc5bb4508"}},{"code_sha256_prefix":"995b95c68ca8681a","entry":"format_wordnet_synset","repo":"ningyuxu/tip_of_tongue","repo_kind":"listed","path":"exps/corpus/wordnet.py","file_url":"https://github.com/ningyuxu/tip_of_tongue/blob/HEAD/exps/corpus/wordnet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"995b95c68ca8681a"}},{"code_sha256_prefix":"f273d6bf63dd6913","entry":"gather","repo":"ningyuxu/tip_of_tongue","repo_kind":"listed","path":"calf/utils/distributed.py","file_url":"https://github.com/ningyuxu/tip_of_tongue/blob/HEAD/calf/utils/distributed.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f273d6bf63dd6913"}},{"code_sha256_prefix":"ed5648c3ddfb07b5","entry":"get_free_gpus","repo":"ningyuxu/tip_of_tongue","repo_kind":"listed","path":"calf/utils/cuda.py","file_url":"https://github.com/ningyuxu/tip_of_tongue/blob/HEAD/calf/utils/cuda.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ed5648c3ddfb07b5"}},{"code_sha256_prefix":"b477f3460d3ffc77","entry":"get_free_gpus_by_gid","repo":"ningyuxu/tip_of_tongue","repo_kind":"listed","path":"calf/utils/cuda.py","file_url":"https://github.com/ningyuxu/tip_of_tongue/blob/HEAD/calf/utils/cuda.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b477f3460d3ffc77"}},{"code_sha256_prefix":"a8da323a33fb2f4d","entry":"reduce","repo":"ningyuxu/tip_of_tongue","repo_kind":"listed","path":"calf/utils/distributed.py","file_url":"https://github.com/ningyuxu/tip_of_tongue/blob/HEAD/calf/utils/distributed.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a8da323a33fb2f4d"}},{"code_sha256_prefix":"1a3fe5eb49125104","entry":"wordnet_synset_data","repo":"ningyuxu/tip_of_tongue","repo_kind":"listed","path":"exps/corpus/wordnet.py","file_url":"https://github.com/ningyuxu/tip_of_tongue/blob/HEAD/exps/corpus/wordnet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1a3fe5eb49125104"}},{"code_sha256_prefix":"904d581818c29bc8","entry":"wordnet_word_dataset","repo":"ningyuxu/tip_of_tongue","repo_kind":"listed","path":"exps/corpus/wordnet.py","file_url":"https://github.com/ningyuxu/tip_of_tongue/blob/HEAD/exps/corpus/wordnet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"904d581818c29bc8"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}