{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/fedmeki-a-benchmark-for-scaling-medical","title":"FEDMEKI: A Benchmark for Scaling Medical Foundation Models via Federated Knowledge Injection","arxiv_id":"2408.09227","date":"2024-08-17","proceeding":null,"authors":["Jiaqi Wang","Xiaochen Wang","Lingjuan Lyu","Jinghui Chen","Fenglong Ma"],"abstract":"This study introduces the Federated Medical Knowledge Injection (FEDMEKI) platform, a new benchmark designed to address the unique challenges of integrating medical knowledge into foundation models under privacy constraints. By leveraging a cross-silo federated learning approach, FEDMEKI circumvents the issues associated with centralized data collection, which is often prohibited under health regulations like the Health Insurance Portability and Accountability Act (HIPAA) in the USA. The platform is meticulously designed to handle multi-site, multi-modal, and multi-task medical data, which includes 7 medical modalities, including images, signals, texts, laboratory test results, vital signs, input variables, and output variables. The curated dataset to validate FEDMEKI covers 8 medical tasks, including 6 classification tasks (lung opacity detection, COVID-19 detection, electrocardiogram (ECG) abnormal detection, mortality prediction, sepsis prediction, and enlarged cardiomediastinum detection) and 2 generation tasks (medical visual question answering (MedVQA) and ECG noise clarification). This comprehensive dataset is partitioned across several clients to facilitate the decentralized training process under 16 benchmark approaches. FEDMEKI not only preserves data privacy but also enhances the capability of medical foundation models by allowing them to learn from a broader spectrum of medical knowledge without direct data exposure, thereby setting a new benchmark in the application of foundation models within the healthcare sector.","url_abs":"https://arxiv.org/abs/2408.09227v1","url_pdf":"https://arxiv.org/pdf/2408.09227v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"fedmeki-a-benchmark-for-scaling-medical","repo_url":"https://github.com/psudslab/FEDMEKI","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"federated-learning","task_name":"Federated Learning"},{"task_slug":"medical-visual-question-answering","task_name":"Medical Visual Question Answering"},{"task_slug":"mortality-prediction","task_name":"Mortality Prediction"},{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"visual-question-answering-1","task_name":"Visual Question Answering"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2408.09227","atlas_url":"https://app.syntology.ai/?focus=2408.09227","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2408.09227"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/psudslab/FEDMEKI","reach":{"status":"ok"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/Jwoo5/ecg-qa","reach":{"status":"ok","spdx":"CC-BY-4.0"}}],"summary":{"ran":8},"by_repo_kind":{"official":{"samples":7,"ran":7,"repositories":1},"found_in_text":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":8,"samples":[{"code_sha256_prefix":"4c7b6b7d100843c7","entry":"build_evaluator","repo":"psudslab/FEDMEKI","repo_kind":"official","path":"FedMEKI/src/ChEF/evaluator.py","file_url":"https://github.com/psudslab/FEDMEKI/blob/HEAD/FedMEKI/src/ChEF/evaluator.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4c7b6b7d100843c7"}},{"code_sha256_prefix":"5129eae6a80ce5f9","entry":"clip","repo":"psudslab/FEDMEKI","repo_kind":"official","path":"FedMEKI/src/datasets/dataset.py","file_url":"https://github.com/psudslab/FEDMEKI/blob/HEAD/FedMEKI/src/datasets/dataset.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5129eae6a80ce5f9"}},{"code_sha256_prefix":"eacef47951a1c344","entry":"get_ptbxl_data_path","repo":"Jwoo5/ecg-qa","repo_kind":"found_in_text","path":"mapping_ptbxl_samples.py","file_url":"https://github.com/Jwoo5/ecg-qa/blob/HEAD/mapping_ptbxl_samples.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"CC-BY-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"eacef47951a1c344"}},{"code_sha256_prefix":"78e54f470017d052","entry":"is_number","repo":"psudslab/FEDMEKI","repo_kind":"official","path":"FedMEKI/src/datasets/utils.py","file_url":"https://github.com/psudslab/FEDMEKI/blob/HEAD/FedMEKI/src/datasets/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"78e54f470017d052"}},{"code_sha256_prefix":"b80626b4df041fc1","entry":"load_Omni","repo":"psudslab/FEDMEKI","repo_kind":"official","path":"FedMEKI/src/ChEF/data_process/Omnibenchmark.py","file_url":"https://github.com/psudslab/FEDMEKI/blob/HEAD/FedMEKI/src/ChEF/data_process/Omnibenchmark.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b80626b4df041fc1"}},{"code_sha256_prefix":"fb66043ff5364d4a","entry":"load_yaml","repo":"psudslab/FEDMEKI","repo_kind":"official","path":"FedMEKI/src/ChEF/evaluator.py","file_url":"https://github.com/psudslab/FEDMEKI/blob/HEAD/FedMEKI/src/ChEF/evaluator.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fb66043ff5364d4a"}},{"code_sha256_prefix":"c08da74e57caf534","entry":"parse_num","repo":"psudslab/FEDMEKI","repo_kind":"official","path":"FedMEKI/src/datasets/utils.py","file_url":"https://github.com/psudslab/FEDMEKI/blob/HEAD/FedMEKI/src/datasets/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c08da74e57caf534"}},{"code_sha256_prefix":"a93d0c45daff9074","entry":"sample_dataset","repo":"psudslab/FEDMEKI","repo_kind":"official","path":"FedMEKI/src/ChEF/evaluator.py","file_url":"https://github.com/psudslab/FEDMEKI/blob/HEAD/FedMEKI/src/ChEF/evaluator.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a93d0c45daff9074"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}