{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/benchmarking-semi-supervised-federated","title":"Improving Semi-supervised Federated Learning by Reducing the Gradient Diversity of Models","arxiv_id":"2008.11364","date":"2020-08-26","proceeding":null,"authors":["Zhengming Zhang","Yaoqing Yang","Zhewei Yao","Yujun Yan","Joseph E. Gonzalez","Michael W. Mahoney"],"abstract":"Federated learning (FL) is a promising way to use the computing power of mobile devices while maintaining the privacy of users. Current work in FL, however, makes the unrealistic assumption that the users have ground-truth labels on their devices, while also assuming that the server has neither data nor labels. In this work, we consider the more realistic scenario where the users have only unlabeled data, while the server has some labeled data, and where the amount of labeled data is smaller than the amount of unlabeled data. We call this learning problem semi-supervised federated learning (SSFL). For SSFL, we demonstrate that a critical issue that affects the test accuracy is the large gradient diversity of the models from different users. Based on this, we investigate several design choices. First, we find that the so-called consistency regularization loss (CRL), which is widely used in semi-supervised learning, performs reasonably well but has large gradient diversity. Second, we find that Batch Normalization (BN) increases gradient diversity. Replacing BN with the recently-proposed Group Normalization (GN) can reduce gradient diversity and improve test accuracy. Third, we show that CRL combined with GN still has a large gradient diversity when the number of users is large. Based on these results, we propose a novel grouping-based model averaging method to replace the FedAvg averaging method. Overall, our grouping-based averaging, combined with GN and CRL, achieves better test accuracy than not just a contemporary paper on SSFL in the same settings (>10\\%), but also four supervised FL algorithms.","url_abs":"https://arxiv.org/abs/2008.11364v2","url_pdf":"https://arxiv.org/pdf/2008.11364v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"benchmarking-semi-supervised-federated","repo_url":"https://github.com/jhcknzzm/SSFL-Benchmarking-Semi-supervised-Federated-Learning","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"diversity","task_name":"Diversity"},{"task_slug":"federated-learning","task_name":"Federated Learning"}],"methods":[{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"group-normalization","method_name":"Group Normalization"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2008.11364","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2008.11364"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jhcknzzm/SSFL-Benchmarking-Semi-supervised-Federated-Learning","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":1,"ran_honours":2,"ran_draft_wrong":1,"unverified":7},"by_repo_kind":{"official":{"samples":11,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"c7442eb6f91cb5bd","entry":"accuracy","repo":"jhcknzzm/SSFL-Benchmarking-Semi-supervised-Federated-Learning","repo_kind":"official","path":"models/base.py","file_url":"https://github.com/jhcknzzm/SSFL-Benchmarking-Semi-supervised-Federated-Learning/blob/HEAD/models/base.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c7442eb6f91cb5bd"}},{"code_sha256_prefix":"e71c2a2bc2fc8909","entry":"compute_conv_output_size","repo":"jhcknzzm/SSFL-Benchmarking-Semi-supervised-Federated-Learning","repo_kind":"official","path":"models/Semi_net.py","file_url":"https://github.com/jhcknzzm/SSFL-Benchmarking-Semi-supervised-Federated-Learning/blob/HEAD/models/Semi_net.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e71c2a2bc2fc8909"}},{"code_sha256_prefix":"de96e9b005b53e0f","entry":"flatten_tensors","repo":"jhcknzzm/SSFL-Benchmarking-Semi-supervised-Federated-Learning","repo_kind":"official","path":"comm_helpers.py","file_url":"https://github.com/jhcknzzm/SSFL-Benchmarking-Semi-supervised-Federated-Learning/blob/HEAD/comm_helpers.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"de96e9b005b53e0f"}},{"code_sha256_prefix":"349c36e737eadfe1","entry":"unflatten_tensors","repo":"jhcknzzm/SSFL-Benchmarking-Semi-supervised-Federated-Learning","repo_kind":"official","path":"comm_helpers.py","file_url":"https://github.com/jhcknzzm/SSFL-Benchmarking-Semi-supervised-Federated-Learning/blob/HEAD/comm_helpers.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"349c36e737eadfe1"}},{"code_sha256_prefix":"f901f7345452fe78","entry":"Load_Avgmodel_weights","repo":"jhcknzzm/SSFL-Benchmarking-Semi-supervised-Federated-Learning","repo_kind":"official","path":"Grad_Diff.py","file_url":"https://github.com/jhcknzzm/SSFL-Benchmarking-Semi-supervised-Federated-Learning/blob/HEAD/Grad_Diff.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f901f7345452fe78"}},{"code_sha256_prefix":"8c55f4031b35b37a","entry":"Load_model_grad_checkpoint","repo":"jhcknzzm/SSFL-Benchmarking-Semi-supervised-Federated-Learning","repo_kind":"official","path":"Grad_Diff.py","file_url":"https://github.com/jhcknzzm/SSFL-Benchmarking-Semi-supervised-Federated-Learning/blob/HEAD/Grad_Diff.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8c55f4031b35b37a"}},{"code_sha256_prefix":"3c7852aeade0ad78","entry":"accuracy","repo":"jhcknzzm/SSFL-Benchmarking-Semi-supervised-Federated-Learning","repo_kind":"official","path":"models/resnet9.py","file_url":"https://github.com/jhcknzzm/SSFL-Benchmarking-Semi-supervised-Federated-Learning/blob/HEAD/models/resnet9.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3c7852aeade0ad78"}},{"code_sha256_prefix":"7dbc4e9dd1b3e1a1","entry":"communicate","repo":"jhcknzzm/SSFL-Benchmarking-Semi-supervised-Federated-Learning","repo_kind":"official","path":"comm_helpers.py","file_url":"https://github.com/jhcknzzm/SSFL-Benchmarking-Semi-supervised-Federated-Learning/blob/HEAD/comm_helpers.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7dbc4e9dd1b3e1a1"}},{"code_sha256_prefix":"b83c8090ce555da4","entry":"conv_bn_relu_pool","repo":"jhcknzzm/SSFL-Benchmarking-Semi-supervised-Federated-Learning","repo_kind":"official","path":"models/resnet9.py","file_url":"https://github.com/jhcknzzm/SSFL-Benchmarking-Semi-supervised-Federated-Learning/blob/HEAD/models/resnet9.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b83c8090ce555da4"}},{"code_sha256_prefix":"f8be03d81632009c","entry":"get","repo":"jhcknzzm/SSFL-Benchmarking-Semi-supervised-Federated-Learning","repo_kind":"official","path":"models/cifar.py","file_url":"https://github.com/jhcknzzm/SSFL-Benchmarking-Semi-supervised-Federated-Learning/blob/HEAD/models/cifar.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f8be03d81632009c"}},{"code_sha256_prefix":"068a61460ff86dc6","entry":"get_groups","repo":"jhcknzzm/SSFL-Benchmarking-Semi-supervised-Federated-Learning","repo_kind":"official","path":"Grad_Diff.py","file_url":"https://github.com/jhcknzzm/SSFL-Benchmarking-Semi-supervised-Federated-Learning/blob/HEAD/Grad_Diff.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"068a61460ff86dc6"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}