{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/fedrolex-model-heterogeneous-federated","title":"FedRolex: Model-Heterogeneous Federated Learning with Rolling Sub-Model Extraction","arxiv_id":"2212.01548","date":"2022-12-03","proceeding":null,"authors":["Samiul Alam","Luyang Liu","Ming Yan","Mi Zhang"],"abstract":"Most cross-device federated learning (FL) studies focus on the model-homogeneous setting where the global server model and local client models are identical. However, such constraint not only excludes low-end clients who would otherwise make unique contributions to model training but also restrains clients from training large models due to on-device resource bottlenecks. In this work, we propose FedRolex, a partial training (PT)-based approach that enables model-heterogeneous FL and can train a global server model larger than the largest client model. At its core, FedRolex employs a rolling sub-model extraction scheme that allows different parts of the global server model to be evenly trained, which mitigates the client drift induced by the inconsistency between individual client models and server model architectures. We show that FedRolex outperforms state-of-the-art PT-based model-heterogeneous FL methods (e.g. Federated Dropout) and reduces the gap between model-heterogeneous and model-homogeneous FL, especially under the large-model large-dataset regime. In addition, we provide theoretical statistical analysis on its advantage over Federated Dropout and evaluate FedRolex on an emulated real-world device distribution to show that FedRolex can enhance the inclusiveness of FL and boost the performance of low-end devices that would otherwise not benefit from FL. Our code is available at: https://github.com/AIoT-MLSys-Lab/FedRolex","url_abs":"https://arxiv.org/abs/2212.01548v2","url_pdf":"https://arxiv.org/pdf/2212.01548v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"fedrolex-model-heterogeneous-federated","repo_url":"https://github.com/aiot-mlsys-lab/fedrolex","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"fedrolex-model-heterogeneous-federated","repo_url":"https://github.com/msu-mlsys-lab/fedrolex","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null},{"paper_slug":"fedrolex-model-heterogeneous-federated","repo_url":"https://github.com/k1l1/SLT","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"federated-learning","task_name":"Federated Learning"},{"task_slug":"model-extraction","task_name":"Model extraction"},{"task_slug":"model","task_name":"model"}],"methods":[{"method_slug":"dropout","method_name":"Dropout"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2212.01548","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2212.01548"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/k1l1/SLT","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/aiot-mlsys-lab/fedrolex","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/msu-mlsys-lab/fedrolex","reach":null}],"summary":{"ran":2,"ran_draft_wrong":3,"ran_honours":1,"unverified":9},"by_repo_kind":{"official":{"samples":4,"ran":1,"repositories":2},"listed":{"samples":11,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"c0cad3dd46b90b51","entry":"FedAvgDevice","repo":"k1l1/SLT","repo_kind":"listed","path":"algorithms/fedrolex.py","file_url":"https://github.com/k1l1/SLT/blob/HEAD/algorithms/fedrolex.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c0cad3dd46b90b51"}},{"code_sha256_prefix":"ae7e474951511463","entry":"FedAvgEvaluation","repo":"k1l1/SLT","repo_kind":"listed","path":"algorithms/fedrolex.py","file_url":"https://github.com/k1l1/SLT/blob/HEAD/algorithms/fedrolex.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ae7e474951511463"}},{"code_sha256_prefix":"35d5b46e4f624d34","entry":"dict_hash","repo":"k1l1/SLT","repo_kind":"listed","path":"run_fl.py","file_url":"https://github.com/k1l1/SLT/blob/HEAD/run_fl.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"35d5b46e4f624d34"}},{"code_sha256_prefix":"558b018bc1c0f19a","entry":"extract_submodel_resnet_structure","repo":"k1l1/SLT","repo_kind":"listed","path":"utils/fedrolex_submodel.py","file_url":"https://github.com/k1l1/SLT/blob/HEAD/utils/fedrolex_submodel.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"558b018bc1c0f19a"}},{"code_sha256_prefix":"6fd4f7ea4a79121e","entry":"input_collate","repo":"aiot-mlsys-lab/fedrolex","repo_kind":"official","path":"data.py","file_url":"https://github.com/aiot-mlsys-lab/fedrolex/blob/HEAD/data.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6fd4f7ea4a79121e"}},{"code_sha256_prefix":"11c9c63eba295e97","entry":"submodel_indices","repo":"k1l1/SLT","repo_kind":"listed","path":"utils/fedrolex_submodel.py","file_url":"https://github.com/k1l1/SLT/blob/HEAD/utils/fedrolex_submodel.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"11c9c63eba295e97"}},{"code_sha256_prefix":"b34975803f9db84d","entry":"FedAvgServer","repo":"k1l1/SLT","repo_kind":"listed","path":"algorithms/fedrolex.py","file_url":"https://github.com/k1l1/SLT/blob/HEAD/algorithms/fedrolex.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b34975803f9db84d"}},{"code_sha256_prefix":"39db5b4010e71eb9","entry":"FedRolexServer","repo":"k1l1/SLT","repo_kind":"listed","path":"algorithms/fedrolex.py","file_url":"https://github.com/k1l1/SLT/blob/HEAD/algorithms/fedrolex.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"39db5b4010e71eb9"}},{"code_sha256_prefix":"b7c57b09b48aa5d6","entry":"FederatedDropoutServer","repo":"k1l1/SLT","repo_kind":"listed","path":"algorithms/fedrolex.py","file_url":"https://github.com/k1l1/SLT/blob/HEAD/algorithms/fedrolex.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b7c57b09b48aa5d6"}},{"code_sha256_prefix":"c03c7b2e83fd0bcd","entry":"ResnetServerRoll","repo":"msu-mlsys-lab/fedrolex","repo_kind":"official","path":"resnet_server.py","file_url":"https://github.com/msu-mlsys-lab/fedrolex/blob/HEAD/resnet_server.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c03c7b2e83fd0bcd"}},{"code_sha256_prefix":"246b2e9dbd5d9a20","entry":"SubsetDevice","repo":"k1l1/SLT","repo_kind":"listed","path":"algorithms/fedrolex.py","file_url":"https://github.com/k1l1/SLT/blob/HEAD/algorithms/fedrolex.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"246b2e9dbd5d9a20"}},{"code_sha256_prefix":"cd24f922c541136d","entry":"extract_submodel_densenet_structure","repo":"k1l1/SLT","repo_kind":"listed","path":"utils/fedrolex_submodel.py","file_url":"https://github.com/k1l1/SLT/blob/HEAD/utils/fedrolex_submodel.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cd24f922c541136d"}},{"code_sha256_prefix":"1231ff80106c5e8e","entry":"fetch_dataset","repo":"aiot-mlsys-lab/fedrolex","repo_kind":"official","path":"data.py","file_url":"https://github.com/aiot-mlsys-lab/fedrolex/blob/HEAD/data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1231ff80106c5e8e"}},{"code_sha256_prefix":"b81843a16c748c02","entry":"make_node","repo":"aiot-mlsys-lab/fedrolex","repo_kind":"official","path":"datasets/imagenet.py","file_url":"https://github.com/aiot-mlsys-lab/fedrolex/blob/HEAD/datasets/imagenet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b81843a16c748c02"}},{"code_sha256_prefix":"22f89a73c84ff63c","entry":"prepare","repo":"k1l1/SLT","repo_kind":"listed","path":"run_fl.py","file_url":"https://github.com/k1l1/SLT/blob/HEAD/run_fl.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"22f89a73c84ff63c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}