{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/brainfounder-towards-brain-foundation-models","title":"BrainSegFounder: Towards 3D Foundation Models for Neuroimage Segmentation","arxiv_id":"2406.10395","date":"2024-06-14","proceeding":null,"authors":["Joseph Cox","Peng Liu","Skylar E. Stolte","Yunchao Yang","Kang Liu","Kyle B. See","Huiwen Ju","Ruogu Fang"],"abstract":"The burgeoning field of brain health research increasingly leverages artificial intelligence (AI) to interpret and analyze neurological data. This study introduces a novel approach towards the creation of medical foundation models by integrating a large-scale multi-modal magnetic resonance imaging (MRI) dataset derived from 41,400 participants in its own. Our method involves a novel two-stage pretraining approach using vision transformers. The first stage is dedicated to encoding anatomical structures in generally healthy brains, identifying key features such as shapes and sizes of different brain regions. The second stage concentrates on spatial information, encompassing aspects like location and the relative positioning of brain structures. We rigorously evaluate our model, BrainFounder, using the Brain Tumor Segmentation (BraTS) challenge and Anatomical Tracings of Lesions After Stroke v2.0 (ATLAS v2.0) datasets. BrainFounder demonstrates a significant performance gain, surpassing the achievements of the previous winning solutions using fully supervised learning. Our findings underscore the impact of scaling up both the complexity of the model and the volume of unlabeled training data derived from generally healthy brains, which enhances the accuracy and predictive capabilities of the model in complex neuroimaging tasks with MRI. The implications of this research provide transformative insights and practical applications in healthcare and make substantial steps towards the creation of foundation models for Medical AI. Our pretrained models and training code can be found at https://github.com/lab-smile/GatorBrain.","url_abs":"https://arxiv.org/abs/2406.10395v3","url_pdf":"https://arxiv.org/pdf/2406.10395v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"brainfounder-towards-brain-foundation-models","repo_url":"https://github.com/lab-smile/brainsegfounder","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"GPL-3.0"}},{"paper_slug":"brainfounder-towards-brain-foundation-models","repo_url":"https://github.com/lab-smile/gatorbrain","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"gone","observed_at":"2026-09-17","how":"tree_404+repo_404"}}],"tasks":[{"task_slug":"brain-tumor-segmentation","task_name":"Brain Tumor Segmentation"},{"task_slug":"tumor-segmentation","task_name":"Tumor Segmentation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2406.10395","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.10395"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lab-smile/brainsegfounder","reach":{"status":"ok","spdx":"GPL-3.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lab-smile/gatorbrain","reach":{"status":"gone","observed_at":"2026-09-17","how":"tree_404+repo_404"}}],"summary":{"ran":3,"unverified":3},"by_repo_kind":{"official":{"samples":6,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"f8b77c481bd9f5ca","entry":"load_T1T210K_datalist","repo":"lab-smile/brainsegfounder","repo_kind":"official","path":"pretrain/utils/data_utils.py","file_url":"https://github.com/lab-smile/brainsegfounder/blob/HEAD/pretrain/utils/data_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"f8b77c481bd9f5ca"}},{"code_sha256_prefix":"d49b3e5bac5b7fba","entry":"load_T1T210K_mixed_datalist","repo":"lab-smile/brainsegfounder","repo_kind":"official","path":"pretrain/utils/data_utils.py","file_url":"https://github.com/lab-smile/brainsegfounder/blob/HEAD/pretrain/utils/data_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"d49b3e5bac5b7fba"}},{"code_sha256_prefix":"28e6fb0027985860","entry":"load_T1T2matched_datalist","repo":"lab-smile/brainsegfounder","repo_kind":"official","path":"pretrain/utils/data_utils.py","file_url":"https://github.com/lab-smile/brainsegfounder/blob/HEAD/pretrain/utils/data_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"28e6fb0027985860"}},{"code_sha256_prefix":"ca555865c1b08f9d","entry":"aug_rand","repo":"lab-smile/brainsegfounder","repo_kind":"official","path":"pretrain/utils/ops.py","file_url":"https://github.com/lab-smile/brainsegfounder/blob/HEAD/pretrain/utils/ops.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"ca555865c1b08f9d"}},{"code_sha256_prefix":"1c797b7495db41fb","entry":"patch_rand_drop","repo":"lab-smile/brainsegfounder","repo_kind":"official","path":"pretrain/utils/ops.py","file_url":"https://github.com/lab-smile/brainsegfounder/blob/HEAD/pretrain/utils/ops.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"1c797b7495db41fb"}},{"code_sha256_prefix":"43b9a22a22eaa1f5","entry":"rot_rand","repo":"lab-smile/brainsegfounder","repo_kind":"official","path":"pretrain/utils/ops.py","file_url":"https://github.com/lab-smile/brainsegfounder/blob/HEAD/pretrain/utils/ops.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"43b9a22a22eaa1f5"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}