{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/bigbio-a-framework-for-data-centric","title":"BigBIO: A Framework for Data-Centric Biomedical Natural Language Processing","arxiv_id":"2206.15076","date":"2022-06-30","proceeding":null,"authors":["Jason Alan Fries","Leon Weber","Natasha Seelam","Gabriel Altay","Debajyoti Datta","Samuele Garda","Myungsun Kang","Ruisi Su","Wojciech Kusa","Samuel Cahyawijaya","Fabio Barth","Simon Ott","Matthias Samwald","Stephen Bach","Stella Biderman","Mario Sänger","Bo wang","Alison Callahan","Daniel León Periñán","Théo Gigant","Patrick Haller","Jenny Chim","Jose David Posada","John Michael Giorgi","Karthik Rangasai Sivaraman","Marc Pàmies","Marianna Nezhurina","Robert Martin","Michael Cullan","Moritz Freidank","Nathan Dahlberg","Shubhanshu Mishra","Shamik Bose","Nicholas Michio Broad","Yanis Labrak","Shlok S Deshmukh","Sid Kiblawi","Ayush Singh","Minh Chien Vu","Trishala Neeraj","Jonas Golde","Albert Villanova del Moral","Benjamin Beilharz"],"abstract":"Training and evaluating language models increasingly requires the construction of meta-datasets --diverse collections of curated data with clear provenance. Natural language prompting has recently lead to improved zero-shot generalization by transforming existing, supervised datasets into a diversity of novel pretraining tasks, highlighting the benefits of meta-dataset curation. While successful in general-domain text, translating these data-centric approaches to biomedical language modeling remains challenging, as labeled biomedical datasets are significantly underrepresented in popular data hubs. To address this challenge, we introduce BigBIO a community library of 126+ biomedical NLP datasets, currently covering 12 task categories and 10+ languages. BigBIO facilitates reproducible meta-dataset curation via programmatic access to datasets and their metadata, and is compatible with current platforms for prompt engineering and end-to-end few/zero shot language model evaluation. We discuss our process for task schema harmonization, data auditing, contribution guidelines, and outline two illustrative use cases: zero-shot evaluation of biomedical prompts and large-scale, multi-task learning. BigBIO is an ongoing community effort and is available at https://github.com/bigscience-workshop/biomedical","url_abs":"https://arxiv.org/abs/2206.15076v1","url_pdf":"https://arxiv.org/pdf/2206.15076v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"bigbio-a-framework-for-data-centric","repo_url":"https://github.com/bigscience-workshop/biomedical","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"bigbio-a-framework-for-data-centric","repo_url":"https://github.com/allenai/beacon","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"diversity","task_name":"Diversity"},{"task_slug":"language-model-evaluation","task_name":"Language Model Evaluation"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"multi-task-learning","task_name":"Multi-Task Learning"},{"task_slug":"prompt-engineering","task_name":"Prompt Engineering"},{"task_slug":"zero-shot-learning","task_name":"Zero-Shot Learning"},{"task_slug":"zero-shot-generalization","task_name":"Zero-shot Generalization"}],"methods":[{"method_slug":null,"method_name":"Library"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2206.15076","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2206.15076"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/NVIDIA/apex","reach":{"status":"ok","spdx":"BSD-3-Clause"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/bigscience-workshop/biomedical","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/allenai/beacon","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":1},"by_repo_kind":{"found_in_text":{"samples":1,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"d5db567b163238c4","entry":"manual_rms_norm","repo":"NVIDIA/apex","repo_kind":"found_in_text","path":"apex/normalization/fused_layer_norm.py","file_url":"https://github.com/NVIDIA/apex/blob/HEAD/apex/normalization/fused_layer_norm.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"d5db567b163238c4"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}