{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/flamby-datasets-and-benchmarks-for-cross-silo","title":"FLamby: Datasets and Benchmarks for Cross-Silo Federated Learning in Realistic Healthcare Settings","arxiv_id":"2210.04620","date":"2022-10-10","proceeding":null,"authors":["Jean Ogier du Terrail","Samy-Safwan Ayed","Edwige Cyffers","Felix Grimberg","Chaoyang He","Regis Loeb","Paul Mangold","Tanguy Marchand","Othmane Marfoq","Erum Mushtaq","Boris Muzellec","Constantin Philippenko","Santiago Silva","Maria Teleńczuk","Shadi Albarqouni","Salman Avestimehr","Aurélien Bellet","Aymeric Dieuleveut","Martin Jaggi","Sai Praneeth Karimireddy","Marco Lorenzi","Giovanni Neglia","Marc Tommasi","Mathieu Andreux"],"abstract":"Federated Learning (FL) is a novel approach enabling several clients holding sensitive data to collaboratively train machine learning models, without centralizing data. The cross-silo FL setting corresponds to the case of few ($2$--$50$) reliable clients, each holding medium to large datasets, and is typically found in applications such as healthcare, finance, or industry. While previous works have proposed representative datasets for cross-device FL, few realistic healthcare cross-silo FL datasets exist, thereby slowing algorithmic research in this critical application. In this work, we propose a novel cross-silo dataset suite focused on healthcare, FLamby (Federated Learning AMple Benchmark of Your cross-silo strategies), to bridge the gap between theory and practice of cross-silo FL. FLamby encompasses 7 healthcare datasets with natural splits, covering multiple tasks, modalities, and data volumes, each accompanied with baseline training code. As an illustration, we additionally benchmark standard FL algorithms on all datasets. Our flexible and modular suite allows researchers to easily download datasets, reproduce results and re-use the different components for their research. FLamby is available at~\\url{www.github.com/owkin/flamby}.","url_abs":"https://arxiv.org/abs/2210.04620v3","url_pdf":"https://arxiv.org/pdf/2210.04620v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"flamby-datasets-and-benchmarks-for-cross-silo","repo_url":"https://github.com/owkin/flamby","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"federated-learning","task_name":"Federated Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2210.04620","atlas_url":"https://app.syntology.ai/?focus=2210.04620","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2210.04620"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/owkin/FLamby","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/owkin/flamby","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":4,"unverified":4},"by_repo_kind":{"official":{"samples":8,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"3e91e64ea676ecce","entry":"evaluate_model_on_tests","repo":"owkin/FLamby","repo_kind":"official","path":"flamby/utils.py","file_url":"https://github.com/owkin/FLamby/blob/HEAD/flamby/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3e91e64ea676ecce"}},{"code_sha256_prefix":"7b172164baf1a3dd","entry":"fill_df_with_xp_results","repo":"owkin/FLamby","repo_kind":"official","path":"flamby/benchmarks/benchmark_utils.py","file_url":"https://github.com/owkin/FLamby/blob/HEAD/flamby/benchmarks/benchmark_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7b172164baf1a3dd"}},{"code_sha256_prefix":"374c91959163c0df","entry":"find_xps_in_df","repo":"owkin/FLamby","repo_kind":"official","path":"flamby/benchmarks/benchmark_utils.py","file_url":"https://github.com/owkin/FLamby/blob/HEAD/flamby/benchmarks/benchmark_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"374c91959163c0df"}},{"code_sha256_prefix":"571f424167465dfc","entry":"init_data_loaders","repo":"owkin/FLamby","repo_kind":"official","path":"flamby/benchmarks/benchmark_utils.py","file_url":"https://github.com/owkin/FLamby/blob/HEAD/flamby/benchmarks/benchmark_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"571f424167465dfc"}},{"code_sha256_prefix":"8715d10e0a857dc9","entry":"check_config","repo":"owkin/FLamby","repo_kind":"official","path":"flamby/benchmarks/conf.py","file_url":"https://github.com/owkin/FLamby/blob/HEAD/flamby/benchmarks/conf.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8715d10e0a857dc9"}},{"code_sha256_prefix":"9f6d33710c9dc368","entry":"get_dataset_args","repo":"owkin/FLamby","repo_kind":"official","path":"flamby/benchmarks/conf.py","file_url":"https://github.com/owkin/FLamby/blob/HEAD/flamby/benchmarks/conf.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9f6d33710c9dc368"}},{"code_sha256_prefix":"76e264c5ebbf5074","entry":"read_config","repo":"owkin/FLamby","repo_kind":"official","path":"flamby/utils.py","file_url":"https://github.com/owkin/FLamby/blob/HEAD/flamby/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"76e264c5ebbf5074"}},{"code_sha256_prefix":"0d3321abd53098b0","entry":"use_gpu_idx","repo":"owkin/FLamby","repo_kind":"official","path":"flamby/gpu_utils.py","file_url":"https://github.com/owkin/FLamby/blob/HEAD/flamby/gpu_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0d3321abd53098b0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}