{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/supernet-training-for-federated-image","title":"Supernet Training for Federated Image Classification under System Heterogeneity","arxiv_id":"2206.01366","date":"2022-06-03","proceeding":null,"authors":["Taehyeon Kim","Se-Young Yun"],"abstract":"Efficient deployment of deep neural networks across many devices and resource constraints, particularly on edge devices, is one of the most challenging problems in the presence of data-privacy preservation issues. Conventional approaches have evolved to either improve a single global model while keeping each local heterogeneous training data decentralized (i.e. data heterogeneity; Federated Learning (FL)) or to train an overarching network that supports diverse architectural settings to address heterogeneous systems equipped with different computational capabilities (i.e. system heterogeneity; Neural Architecture Search). However, few studies have considered both directions simultaneously. This paper proposes the federation of supernet training (FedSup) framework to consider both scenarios simultaneously, i.e., where clients send and receive a supernet that contains all possible architectures sampled from itself. The approach is inspired by observing that averaging parameters during model aggregation for FL is similar to weight-sharing in supernet training. Thus, the proposed FedSup framework combines a weight-sharing approach widely used for training single shot models with FL averaging (FedAvg). Furthermore, we develop an efficient algorithm (E-FedSup) by sending the sub-model to clients on the broadcast stage to reduce communication costs and training overhead, including several strategies to enhance supernet training in the FL environment. We verify the proposed approach with extensive empirical evaluations. The resulting framework also ensures data and model heterogeneity robustness on several standard benchmarks.","url_abs":"https://arxiv.org/abs/2206.01366v5","url_pdf":"https://arxiv.org/pdf/2206.01366v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"supernet-training-for-federated-image","repo_url":"https://github.com/Kthyeon/fedsup","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"classification-1","task_name":"Classification"},{"task_slug":"federated-learning","task_name":"Federated Learning"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"architecture-search","task_name":"Neural Architecture Search"},{"task_slug":"image-classification","task_name":"image-classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2206.01366","atlas_url":"https://app.syntology.ai/?focus=2206.01366","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2206.01366"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Kthyeon/fedsup","reach":null}],"summary":{"ran_draft_wrong":1,"unverified":3},"by_repo_kind":{"official":{"samples":4,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"7de57851e2c68907","entry":"create_model","repo":"Kthyeon/fedsup","repo_kind":"official","path":"models/model_factory.py","file_url":"https://github.com/Kthyeon/fedsup/blob/HEAD/models/model_factory.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7de57851e2c68907"}},{"code_sha256_prefix":"68d19ed82e7c5cee","entry":"get_my_model","repo":"Kthyeon/fedsup","repo_kind":"official","path":"utils/comm.py","file_url":"https://github.com/Kthyeon/fedsup/blob/HEAD/utils/comm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"68d19ed82e7c5cee"}},{"code_sha256_prefix":"0c26f59a316db857","entry":"get_parallel_model","repo":"Kthyeon/fedsup","repo_kind":"official","path":"utils/comm.py","file_url":"https://github.com/Kthyeon/fedsup/blob/HEAD/utils/comm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0c26f59a316db857"}},{"code_sha256_prefix":"0bc73837e6685e49","entry":"reduce_eval_results","repo":"Kthyeon/fedsup","repo_kind":"official","path":"utils/comm.py","file_url":"https://github.com/Kthyeon/fedsup/blob/HEAD/utils/comm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0bc73837e6685e49"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}