{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2607-08219","title":"Benchmark Evaluation of Federated Learning on Multi-organ Images","arxiv_id":"2607.08219","date":"2026-07-09","proceeding":null,"authors":["Junbin Mao","Xu Tian","Jianchun Zhu","Ludi Li","Jin Liu"],"abstract":"The privacy requirements of medical data and its substantial variations across organs and modalities hinder the clinical implementation of medical AI. Federated learning (FL) is a feasible approach to overcome these challenges. Due to the continuous emergence of FL algorithms and the highly heterogeneous nature of medical data, objectively evaluating their performance in real-world clinical settings remains difficult. Therefore, a comprehensive federated medical imaging benchmark, serving as a unified evaluation standard, is crucial for advancing the technology toward reliable clinical application. Existing federated medical imaging benchmarks have not yet adequately incorporated state-of-the-art algorithms, are limited to data from single organs or modalities, and overly emphasize model accuracy, making it difficult to comprehensively assess the overall efficacy of FL in real-world medical environments. To address these challenges, we developed the MobenFL benchmark. This benchmark integrates 20 cutting-edge FL algorithms and 22 medical imaging datasets, covering 12 critical organs across the human body, surpassing existing benchmark in breadth. In terms of evaluation dimensions, MobenFL not only assesses performance but also systematically incorporates key metrics such as algorithmic efficiency and privacy protection capabilities. Additionally, it conducts specialized evaluations for complex real-world clinical scenarios involving different diseases, devices, and imaging modalities, thereby providing a comprehensive and in-depth evaluation framework for the clinical application of FL in the medical field.","url_abs":"https://arxiv.org/abs/2607.08219","url_pdf":"https://arxiv.org/pdf/2607.08219","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2607.08219","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2607.08219"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/yutian0315/MobenFL","reach":null}],"summary":{"ran":6,"ran_fixture":1,"unverified":1},"by_repo_kind":{"found_in_text":{"samples":8,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":8,"samples":[{"code_sha256_prefix":"129664a1e6edf268","entry":"FileType","repo":"yutian0315/MobenFL","repo_kind":"found_in_text","path":"dataloders.py","file_url":"https://github.com/yutian0315/MobenFL/blob/HEAD/dataloders.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"129664a1e6edf268"}},{"code_sha256_prefix":"fb0c3695600fa8f7","entry":"calc_template","repo":"yutian0315/MobenFL","repo_kind":"found_in_text","path":"models/strategy.py","file_url":"https://github.com/yutian0315/MobenFL/blob/HEAD/models/strategy.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fb0c3695600fa8f7"}},{"code_sha256_prefix":"36a01988eb899505","entry":"calculate_accuracy","repo":"yutian0315/MobenFL","repo_kind":"found_in_text","path":"utils.py","file_url":"https://github.com/yutian0315/MobenFL/blob/HEAD/utils.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"36a01988eb899505"}},{"code_sha256_prefix":"598809754d8f0f90","entry":"dataset_iid","repo":"yutian0315/MobenFL","repo_kind":"found_in_text","path":"dataloders.py","file_url":"https://github.com/yutian0315/MobenFL/blob/HEAD/dataloders.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"598809754d8f0f90"}},{"code_sha256_prefix":"231b8ae7d1a37a64","entry":"dirichlet_distribution_split","repo":"yutian0315/MobenFL","repo_kind":"found_in_text","path":"utils.py","file_url":"https://github.com/yutian0315/MobenFL/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"231b8ae7d1a37a64"}},{"code_sha256_prefix":"4e6319daf4416cf6","entry":"get_ep_prior","repo":"yutian0315/MobenFL","repo_kind":"found_in_text","path":"models/strategy.py","file_url":"https://github.com/yutian0315/MobenFL/blob/HEAD/models/strategy.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4e6319daf4416cf6"}},{"code_sha256_prefix":"1c361483c7cc9e34","entry":"preprocess_features","repo":"yutian0315/MobenFL","repo_kind":"found_in_text","path":"utils.py","file_url":"https://github.com/yutian0315/MobenFL/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1c361483c7cc9e34"}},{"code_sha256_prefix":"66011124d9dfd24a","entry":"calc_statistics","repo":"yutian0315/MobenFL","repo_kind":"found_in_text","path":"models/strategy.py","file_url":"https://github.com/yutian0315/MobenFL/blob/HEAD/models/strategy.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"66011124d9dfd24a"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":{"kind":"leaderboard_placements","source":"Syntology's leaderboard-shaped extractor over the paper's own arXiv-HTML tables: a model pointed at a cell, the number was read from that cell and checked against the board's metric, dataset, split and scale, and an independent check accepted the entry; not reviewed by the paper's authors or the archive's editors","extractor_model":"global.anthropic.claude-sonnet-4-5-20250929-v1:0","verifier_model":null,"prompt_sha":"fa63d4bb9d755694","coverage":{"sentence":"Syntology has checked 6,795 of the 9,581 papers on this site that are newer than the archive; results from the others appear after they are checked.","papers_newer_than_archive":9581,"papers_checked":6795},"entries":[],"not_placed":{"boards":0,"rejected_by_independent_check":0,"refused_by_a_rule":0,"check_did_not_answer":0,"proposed_without_a_cell":0,"declined_by_site":0}}}