{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/tct-convexifying-federated-learning-using","title":"TCT: Convexifying Federated Learning using Bootstrapped Neural Tangent Kernels","arxiv_id":"2207.06343","date":"2022-07-13","proceeding":null,"authors":["Yaodong Yu","Alexander Wei","Sai Praneeth Karimireddy","Yi Ma","Michael I. Jordan"],"abstract":"State-of-the-art federated learning methods can perform far worse than their centralized counterparts when clients have dissimilar data distributions. For neural networks, even when centralized SGD easily finds a solution that is simultaneously performant for all clients, current federated optimization methods fail to converge to a comparable solution. We show that this performance disparity can largely be attributed to optimization challenges presented by nonconvexity. Specifically, we find that the early layers of the network do learn useful features, but the final layers fail to make use of them. That is, federated optimization applied to this non-convex problem distorts the learning of the final layers. Leveraging this observation, we propose a Train-Convexify-Train (TCT) procedure to sidestep this issue: first, learn features using off-the-shelf methods (e.g., FedAvg); then, optimize a convexified problem obtained from the network's empirical neural tangent kernel approximation. Our technique yields accuracy improvements of up to +36% on FMNIST and +37% on CIFAR10 when clients have dissimilar data.","url_abs":"https://arxiv.org/abs/2207.06343v2","url_pdf":"https://arxiv.org/pdf/2207.06343v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"tct-convexifying-federated-learning-using","repo_url":"https://github.com/yaodongyu/tct","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"federated-learning","task_name":"Federated Learning"}],"methods":[{"method_slug":"sgd","method_name":"SGD"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2207.06343","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2207.06343"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/yaodongyu/TCT","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yaodongyu/tct","reach":{"status":"ok"}}],"summary":{"ran_fixture":3,"unverified":1},"by_repo_kind":{"official":{"samples":4,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"7583cadd1c91cbe1","entry":"client_update","repo":"yaodongyu/tct","repo_kind":"official","path":"mnist/utils.py","file_url":"https://github.com/yaodongyu/tct/blob/HEAD/mnist/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7583cadd1c91cbe1"}},{"code_sha256_prefix":"8b0f3dbd491e9dab","entry":"evaluate_many_models","repo":"yaodongyu/tct","repo_kind":"official","path":"mnist/utils.py","file_url":"https://github.com/yaodongyu/tct/blob/HEAD/mnist/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8b0f3dbd491e9dab"}},{"code_sha256_prefix":"bca65ccbc3175672","entry":"evaluate_model","repo":"yaodongyu/tct","repo_kind":"official","path":"mnist/utils.py","file_url":"https://github.com/yaodongyu/tct/blob/HEAD/mnist/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bca65ccbc3175672"}},{"code_sha256_prefix":"45eda001027d5cb1","entry":"compute_eNTK","repo":"yaodongyu/tct","repo_kind":"official","path":"mnist/utils.py","file_url":"https://github.com/yaodongyu/tct/blob/HEAD/mnist/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"45eda001027d5cb1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}