{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/federated-learning-with-extremely-noisy","title":"Federated Learning with Extremely Noisy Clients via Negative Distillation","arxiv_id":"2312.12703","date":"2023-12-20","proceeding":null,"authors":["Yang Lu","Lin Chen","Yonggang Zhang","Yiliang Zhang","Bo Han","Yiu-ming Cheung","Hanzi Wang"],"abstract":"Federated learning (FL) has shown remarkable success in cooperatively training deep models, while typically struggling with noisy labels. Advanced works propose to tackle label noise by a re-weighting strategy with a strong assumption, i.e., mild label noise. However, it may be violated in many real-world FL scenarios because of highly contaminated clients, resulting in extreme noise ratios, e.g., $>$90%. To tackle extremely noisy clients, we study the robustness of the re-weighting strategy, showing a pessimistic conclusion: minimizing the weight of clients trained over noisy data outperforms re-weighting strategies. To leverage models trained on noisy clients, we propose a novel approach, called negative distillation (FedNed). FedNed first identifies noisy clients and employs rather than discards the noisy clients in a knowledge distillation manner. In particular, clients identified as noisy ones are required to train models using noisy labels and pseudo-labels obtained by global models. The model trained on noisy labels serves as a `bad teacher' in knowledge distillation, aiming to decrease the risk of providing incorrect information. Meanwhile, the model trained on pseudo-labels is involved in model aggregation if not identified as a noisy client. Consequently, through pseudo-labeling, FedNed gradually increases the trustworthiness of models trained on noisy clients, while leveraging all clients for model aggregation through negative distillation. To verify the efficacy of FedNed, we conduct extensive experiments under various settings, demonstrating that FedNed can consistently outperform baselines and achieve state-of-the-art performance. Our code is available at https://github.com/linChen99/FedNed.","url_abs":"https://arxiv.org/abs/2312.12703v2","url_pdf":"https://arxiv.org/pdf/2312.12703v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"federated-learning-with-extremely-noisy","repo_url":"https://github.com/linchen99/fedned","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"federated-learning","task_name":"Federated Learning"},{"task_slug":"knowledge-distillation","task_name":"Knowledge Distillation"}],"methods":[{"method_slug":"knowledge-distillation","method_name":"Knowledge Distillation"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2312.12703","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2312.12703"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/linchen99/fedned","reach":{"status":"ok"}}],"summary":{"ran":8,"ran_draft_wrong":4,"ran_honours":1,"ran_fixture":1,"unverified":3},"by_repo_kind":{"official":{"samples":17,"ran":14,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":17,"samples":[{"code_sha256_prefix":"c6e7c3451c85404f","entry":"AutoContrast","repo":"linchen99/fedned","repo_kind":"official","path":"dataset/utils/randaugment.py","file_url":"https://github.com/linchen99/fedned/blob/HEAD/dataset/utils/randaugment.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c6e7c3451c85404f"}},{"code_sha256_prefix":"0d1a8d6ce2015fc7","entry":"Brightness","repo":"linchen99/fedned","repo_kind":"official","path":"dataset/utils/randaugment.py","file_url":"https://github.com/linchen99/fedned/blob/HEAD/dataset/utils/randaugment.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0d1a8d6ce2015fc7"}},{"code_sha256_prefix":"32cb3f5760fa593e","entry":"Color","repo":"linchen99/fedned","repo_kind":"official","path":"dataset/utils/randaugment.py","file_url":"https://github.com/linchen99/fedned/blob/HEAD/dataset/utils/randaugment.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"32cb3f5760fa593e"}},{"code_sha256_prefix":"7f1c8cbd622ebc50","entry":"classify_label","repo":"linchen99/fedned","repo_kind":"official","path":"dataset/utils/dataset.py","file_url":"https://github.com/linchen99/fedned/blob/HEAD/dataset/utils/dataset.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7f1c8cbd622ebc50"}},{"code_sha256_prefix":"361c22d6b53b0a22","entry":"client_iid_indices","repo":"linchen99/fedned","repo_kind":"official","path":"dataset/utils/sampling.py","file_url":"https://github.com/linchen99/fedned/blob/HEAD/dataset/utils/sampling.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"361c22d6b53b0a22"}},{"code_sha256_prefix":"95e6e17340d3ad10","entry":"conv1x1","repo":"linchen99/fedned","repo_kind":"official","path":"models/resnet.py","file_url":"https://github.com/linchen99/fedned/blob/HEAD/models/resnet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"95e6e17340d3ad10"}},{"code_sha256_prefix":"794a2ed91a5175bd","entry":"conv3x3","repo":"linchen99/fedned","repo_kind":"official","path":"models/resnet.py","file_url":"https://github.com/linchen99/fedned/blob/HEAD/models/resnet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"794a2ed91a5175bd"}},{"code_sha256_prefix":"2aa5d536ff654fac","entry":"conv3x3","repo":"linchen99/fedned","repo_kind":"official","path":"models/model_feature.py","file_url":"https://github.com/linchen99/fedned/blob/HEAD/models/model_feature.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2aa5d536ff654fac"}},{"code_sha256_prefix":"cc246483ef79cbc8","entry":"get_set_gpus","repo":"linchen99/fedned","repo_kind":"official","path":"utils/tools.py","file_url":"https://github.com/linchen99/fedned/blob/HEAD/utils/tools.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cc246483ef79cbc8"}},{"code_sha256_prefix":"aa07673ccf9afd55","entry":"label_indices2indices","repo":"linchen99/fedned","repo_kind":"official","path":"dataset/utils/dataset.py","file_url":"https://github.com/linchen99/fedned/blob/HEAD/dataset/utils/dataset.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"aa07673ccf9afd55"}},{"code_sha256_prefix":"4c7f4834de996b2f","entry":"noisify_label","repo":"linchen99/fedned","repo_kind":"official","path":"dataset/utils/noisify.py","file_url":"https://github.com/linchen99/fedned/blob/HEAD/dataset/utils/noisify.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4c7f4834de996b2f"}},{"code_sha256_prefix":"d4682009629cffa4","entry":"norm2d","repo":"linchen99/fedned","repo_kind":"official","path":"models/model_feature.py","file_url":"https://github.com/linchen99/fedned/blob/HEAD/models/model_feature.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d4682009629cffa4"}},{"code_sha256_prefix":"1e2e488b6e0732b4","entry":"partition_balance","repo":"linchen99/fedned","repo_kind":"official","path":"dataset/utils/sampling.py","file_url":"https://github.com/linchen99/fedned/blob/HEAD/dataset/utils/sampling.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1e2e488b6e0732b4"}},{"code_sha256_prefix":"d810c1d73375ec4c","entry":"partition_train_teach","repo":"linchen99/fedned","repo_kind":"official","path":"dataset/utils/dataset.py","file_url":"https://github.com/linchen99/fedned/blob/HEAD/dataset/utils/dataset.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d810c1d73375ec4c"}},{"code_sha256_prefix":"d5aaade96b475879","entry":"clients_non_iid_indices","repo":"linchen99/fedned","repo_kind":"official","path":"dataset/utils/sampling.py","file_url":"https://github.com/linchen99/fedned/blob/HEAD/dataset/utils/sampling.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d5aaade96b475879"}},{"code_sha256_prefix":"3e4ccc9c4bfaf8a8","entry":"get_cifar10","repo":"linchen99/fedned","repo_kind":"official","path":"dataset/get_cifar10.py","file_url":"https://github.com/linchen99/fedned/blob/HEAD/dataset/get_cifar10.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3e4ccc9c4bfaf8a8"}},{"code_sha256_prefix":"aa4aa861ab25b3fb","entry":"resnet18","repo":"linchen99/fedned","repo_kind":"official","path":"models/resnet.py","file_url":"https://github.com/linchen99/fedned/blob/HEAD/models/resnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"aa4aa861ab25b3fb"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}