{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/fjord-fair-and-accurate-federated-learning","title":"FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropout","arxiv_id":"2102.13451","date":"2021-02-26","proceeding":"NeurIPS 2021 12","authors":["Samuel Horvath","Stefanos Laskaridis","Mario Almeida","Ilias Leontiadis","Stylianos I. Venieris","Nicholas D. Lane"],"abstract":"Federated Learning (FL) has been gaining significant traction across different ML tasks, ranging from vision to keyboard predictions. In large-scale deployments, client heterogeneity is a fact and constitutes a primary problem for fairness, training performance and accuracy. Although significant efforts have been made into tackling statistical data heterogeneity, the diversity in the processing capabilities and network bandwidth of clients, termed as system heterogeneity, has remained largely unexplored. Current solutions either disregard a large portion of available devices or set a uniform limit on the model's capacity, restricted by the least capable participants. In this work, we introduce Ordered Dropout, a mechanism that achieves an ordered, nested representation of knowledge in deep neural networks (DNNs) and enables the extraction of lower footprint submodels without the need of retraining. We further show that for linear maps our Ordered Dropout is equivalent to SVD. We employ this technique, along with a self-distillation methodology, in the realm of FL in a framework called FjORD. FjORD alleviates the problem of client system heterogeneity by tailoring the model width to the client's capabilities. Extensive evaluation on both CNNs and RNNs across diverse modalities shows that FjORD consistently leads to significant performance gains over state-of-the-art baselines, while maintaining its nested structure.","url_abs":"https://arxiv.org/abs/2102.13451v5","url_pdf":"https://arxiv.org/pdf/2102.13451v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"fjord-fair-and-accurate-federated-learning","repo_url":"https://github.com/adap/flower","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"tf","reach":null},{"paper_slug":"fjord-fair-and-accurate-federated-learning","repo_url":"https://github.com/k1l1/SLT","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"fairness","task_name":"Fairness"},{"task_slug":"federated-learning","task_name":"Federated Learning"}],"methods":[{"method_slug":"dropout","method_name":"Dropout"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2102.13451","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2102.13451"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/k1l1/SLT","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/adap/flower","reach":null}],"summary":{"ran":2,"unverified":1},"by_repo_kind":{"listed":{"samples":3,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"0b51c806bdc788bc","entry":"FjordDevice","repo":"k1l1/SLT","repo_kind":"listed","path":"algorithms/fjord.py","file_url":"https://github.com/k1l1/SLT/blob/HEAD/algorithms/fjord.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0b51c806bdc788bc"}},{"code_sha256_prefix":"e37de38d95d7cf93","entry":"SubsetDevice","repo":"k1l1/SLT","repo_kind":"listed","path":"algorithms/fjord.py","file_url":"https://github.com/k1l1/SLT/blob/HEAD/algorithms/fjord.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e37de38d95d7cf93"}},{"code_sha256_prefix":"ed316958b64c1cb2","entry":"FedAvgDevice","repo":"k1l1/SLT","repo_kind":"listed","path":"algorithms/fjord.py","file_url":"https://github.com/k1l1/SLT/blob/HEAD/algorithms/fjord.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ed316958b64c1cb2"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}