{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/federated-learning-on-non-iid-data-silos-an","title":"Federated Learning on Non-IID Data Silos: An Experimental Study","arxiv_id":"2102.02079","date":"2021-02-03","proceeding":null,"authors":["Qinbin Li","Yiqun Diao","Quan Chen","Bingsheng He"],"abstract":"Due to the increasing privacy concerns and data regulations, training data have been increasingly fragmented, forming distributed databases of multiple \"data silos\" (e.g., within different organizations and countries). To develop effective machine learning services, there is a must to exploit data from such distributed databases without exchanging the raw data. Recently, federated learning (FL) has been a solution with growing interests, which enables multiple parties to collaboratively train a machine learning model without exchanging their local data. A key and common challenge on distributed databases is the heterogeneity of the data distribution among the parties. The data of different parties are usually non-independently and identically distributed (i.e., non-IID). There have been many FL algorithms to address the learning effectiveness under non-IID data settings. However, there lacks an experimental study on systematically understanding their advantages and disadvantages, as previous studies have very rigid data partitioning strategies among parties, which are hardly representative and thorough. In this paper, to help researchers better understand and study the non-IID data setting in federated learning, we propose comprehensive data partitioning strategies to cover the typical non-IID data cases. Moreover, we conduct extensive experiments to evaluate state-of-the-art FL algorithms. We find that non-IID does bring significant challenges in learning accuracy of FL algorithms, and none of the existing state-of-the-art FL algorithms outperforms others in all cases. Our experiments provide insights for future studies of addressing the challenges in \"data silos\".","url_abs":"https://arxiv.org/abs/2102.02079v4","url_pdf":"https://arxiv.org/pdf/2102.02079v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"federated-learning-on-non-iid-data-silos-an","repo_url":"https://github.com/Xtra-Computing/NIID-Bench","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"federated-learning-on-non-iid-data-silos-an","repo_url":"https://github.com/eleanor-w/kci_for_fl","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"federated-learning-on-non-iid-data-silos-an","repo_url":"https://github.com/ituvisionlab/bfl-p","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"federated-learning-on-non-iid-data-silos-an","repo_url":"https://github.com/adap/flower","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"tf","reach":null}],"tasks":[{"task_slug":"machine-learning","task_name":"BIG-bench Machine Learning"},{"task_slug":"federated-learning","task_name":"Federated Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2102.02079","atlas_url":"https://app.syntology.ai/?focus=2102.02079","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2102.02079"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/adap/flower","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/eleanor-w/kci_for_fl","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ituvisionlab/bfl-p","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Xtra-Computing/NIID-Bench","reach":null}],"summary":{"ran_draft_wrong":1,"unverified":3},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1},"listed":{"samples":3,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"82a6908178b4dfc0","entry":"partition_data","repo":"Xtra-Computing/NIID-Bench","repo_kind":"official","path":"partition.py","file_url":"https://github.com/Xtra-Computing/NIID-Bench/blob/HEAD/partition.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"82a6908178b4dfc0"}},{"code_sha256_prefix":"bf68b2cfab5826f0","entry":"init_nets","repo":"eleanor-w/kci_for_fl","repo_kind":"listed","path":"experiments.py","file_url":"https://github.com/eleanor-w/kci_for_fl/blob/HEAD/experiments.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bf68b2cfab5826f0"}},{"code_sha256_prefix":"54a7795b20263264","entry":"train_net","repo":"eleanor-w/kci_for_fl","repo_kind":"listed","path":"experiments.py","file_url":"https://github.com/eleanor-w/kci_for_fl/blob/HEAD/experiments.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"54a7795b20263264"}},{"code_sha256_prefix":"c145a7562ae9d57c","entry":"train_net_fedprox","repo":"eleanor-w/kci_for_fl","repo_kind":"listed","path":"experiments.py","file_url":"https://github.com/eleanor-w/kci_for_fl/blob/HEAD/experiments.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c145a7562ae9d57c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}