{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/alephbert-a-hebrew-large-pre-trained-language","title":"AlephBERT:A Hebrew Large Pre-Trained Language Model to Start-off your Hebrew NLP Application With","arxiv_id":"2104.04052","date":"2021-04-08","proceeding":null,"authors":["Amit Seker","Elron Bandel","Dan Bareket","Idan Brusilovsky","Refael Shaked Greenfeld","Reut Tsarfaty"],"abstract":"Large Pre-trained Language Models (PLMs) have become ubiquitous in the development of language understanding technology and lie at the heart of many artificial intelligence advances. While advances reported for English using PLMs are unprecedented, reported advances using PLMs in Hebrew are few and far between. The problem is twofold. First, Hebrew resources available for training NLP models are not at the same order of magnitude as their English counterparts. Second, there are no accepted tasks and benchmarks to evaluate the progress of Hebrew PLMs on. In this work we aim to remedy both aspects. First, we present AlephBERT, a large pre-trained language model for Modern Hebrew, which is trained on larger vocabulary and a larger dataset than any Hebrew PLM before. Second, using AlephBERT we present new state-of-the-art results on multiple Hebrew tasks and benchmarks, including: Segmentation, Part-of-Speech Tagging, full Morphological Tagging, Named-Entity Recognition and Sentiment Analysis. We make our AlephBERT model publicly available, providing a single point of entry for the development of Hebrew NLP applications.","url_abs":"https://arxiv.org/abs/2104.04052v1","url_pdf":"https://arxiv.org/pdf/2104.04052v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"alephbert-a-hebrew-large-pre-trained-language","repo_url":"https://github.com/OnlpLab/Hebrew-Sentiment-Data","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"alephbert-a-hebrew-large-pre-trained-language","repo_url":"https://github.com/8400thehealthnetwork/hebsafeharbor","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"morphological-tagging","task_name":"Morphological Tagging"},{"task_slug":"named-entity-recognition-1","task_name":"Named Entity Recognition"},{"task_slug":"named-entity-recognition-ner","task_name":"Named Entity Recognition (NER)"},{"task_slug":"part-of-speech-tagging","task_name":"Part-Of-Speech Tagging"},{"task_slug":"sentiment-analysis","task_name":"Sentiment Analysis"},{"task_slug":"named-entity-recognition","task_name":"named-entity-recognition"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/named-entity-recognition-on-nemo-corpus-morph-1","task":"Named Entity Recognition (NER)","dataset":"NEMO-Corpus (morph,test)","model":"AlephBERT-base Pipeline","rank_in_archive_order":1,"of":3,"metrics":{"F1":"80.15"},"uses_additional_data":false},{"leaderboard":"/sota/named-entity-recognition-on-nemo-corpus-morph-1","task":"Named Entity Recognition (NER)","dataset":"NEMO-Corpus (morph,test)","model":"AlephBERT-base MultiTask","rank_in_archive_order":2,"of":3,"metrics":{"F1":"79.15"},"uses_additional_data":false},{"leaderboard":"/sota/named-entity-recognition-on-nemo-corpus-token-1","task":"Named Entity Recognition (NER)","dataset":"NEMO-Corpus (token,test)","model":"AlephBERT-base","rank_in_archive_order":1,"of":2,"metrics":{"F1":"84.91"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2104.04052","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2104.04052"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/8400thehealthnetwork/hebsafeharbor","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/OnlpLab/Hebrew-Sentiment-Data","reach":{"status":"ok"}}],"summary":{"ran":2},"by_repo_kind":{"listed":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"55094f94d4dbb4c9","entry":"set_date_mention_component","repo":"8400thehealthnetwork/hebsafeharbor","repo_kind":"listed","path":"hebsafeharbor/common/date_utils.py","file_url":"https://github.com/8400thehealthnetwork/hebsafeharbor/blob/HEAD/hebsafeharbor/common/date_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"55094f94d4dbb4c9"}},{"code_sha256_prefix":"5b7e8295decb30dc","entry":"set_date_mention_from_pattern_group_names","repo":"8400thehealthnetwork/hebsafeharbor","repo_kind":"listed","path":"hebsafeharbor/common/date_utils.py","file_url":"https://github.com/8400thehealthnetwork/hebsafeharbor/blob/HEAD/hebsafeharbor/common/date_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5b7e8295decb30dc"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}