{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/decoupled-greedy-learning-of-cnns","title":"Decoupled Greedy Learning of CNNs","arxiv_id":"1901.08164","date":"2019-01-23","proceeding":"ICML 2020 1","authors":["Eugene Belilovsky","Michael Eickenberg","Edouard Oyallon"],"abstract":"A commonly cited inefficiency of neural network training by back-propagation is the update locking problem: each layer must wait for the signal to propagate through the full network before updating. Several alternatives that can alleviate this issue have been proposed. In this context, we consider a simpler, but more effective, substitute that uses minimal feedback, which we call Decoupled Greedy Learning (DGL). It is based on a greedy relaxation of the joint training objective, recently shown to be effective in the context of Convolutional Neural Networks (CNNs) on large-scale image classification. We consider an optimization of this objective that permits us to decouple the layer training, allowing for layers or modules in networks to be trained with a potentially linear parallelization in layers. With the use of a replay buffer we show this approach can be extended to asynchronous settings, where modules can operate with possibly large communication delays. We show theoretically and empirically that this approach converges. Then, we empirically find that it can lead to better generalization than sequential greedy optimization. We demonstrate the effectiveness of DGL against alternative approaches on the CIFAR-10 dataset and on the large-scale ImageNet dataset.","url_abs":"https://arxiv.org/abs/1901.08164v4","url_pdf":"https://arxiv.org/pdf/1901.08164v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"decoupled-greedy-learning-of-cnns","repo_url":"https://github.com/eugenium/DGL","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"decoupled-greedy-learning-of-cnns","repo_url":"https://github.com/batuozt/gleam","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"image-classification","task_name":"image-classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1901.08164","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1901.08164"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/batuozt/gleam","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/eugenium/DGL","reach":null}],"summary":{"ran":2,"ran_honours":1,"unverified":11},"by_repo_kind":{"official":{"samples":2,"ran":1,"repositories":1},"listed":{"samples":12,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"e8526a516b4f9206","entry":"convert_basic_c2_names","repo":"batuozt/gleam","repo_kind":"listed","path":"ss_recon/checkpoint/c2_model_loading.py","file_url":"https://github.com/batuozt/gleam/blob/HEAD/ss_recon/checkpoint/c2_model_loading.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e8526a516b4f9206"}},{"code_sha256_prefix":"52810b504f0d1066","entry":"convert_c2_detectron_names","repo":"batuozt/gleam","repo_kind":"listed","path":"ss_recon/checkpoint/c2_model_loading.py","file_url":"https://github.com/batuozt/gleam/blob/HEAD/ss_recon/checkpoint/c2_model_loading.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"52810b504f0d1066"}},{"code_sha256_prefix":"f81566b35c748e30","entry":"lr_scheduler","repo":"eugenium/DGL","repo_kind":"official","path":"dni_comparisons/cifar_cnn_dni.py","file_url":"https://github.com/eugenium/DGL/blob/HEAD/dni_comparisons/cifar_cnn_dni.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f81566b35c748e30"}},{"code_sha256_prefix":"9d2d1a47f9b68657","entry":"approximately_split","repo":"batuozt/gleam","repo_kind":"listed","path":"datasets/utils/data_partition.py","file_url":"https://github.com/batuozt/gleam/blob/HEAD/datasets/utils/data_partition.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9d2d1a47f9b68657"}},{"code_sha256_prefix":"6ca55eea797cb3a8","entry":"approximately_split_weighted","repo":"batuozt/gleam","repo_kind":"listed","path":"datasets/utils/data_partition.py","file_url":"https://github.com/batuozt/gleam/blob/HEAD/datasets/utils/data_partition.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6ca55eea797cb3a8"}},{"code_sha256_prefix":"cc12efe97cafdfd8","entry":"fftc","repo":"batuozt/gleam","repo_kind":"listed","path":"datasets/utils/fftc.py","file_url":"https://github.com/batuozt/gleam/blob/HEAD/datasets/utils/fftc.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cc12efe97cafdfd8"}},{"code_sha256_prefix":"fdc31294150d9ef4","entry":"fftnc","repo":"batuozt/gleam","repo_kind":"listed","path":"datasets/utils/fftc.py","file_url":"https://github.com/batuozt/gleam/blob/HEAD/datasets/utils/fftc.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fdc31294150d9ef4"}},{"code_sha256_prefix":"4caeef87f87a4bb7","entry":"find_wandb_exp_id","repo":"batuozt/gleam","repo_kind":"listed","path":"ss_recon/engine/defaults.py","file_url":"https://github.com/batuozt/gleam/blob/HEAD/ss_recon/engine/defaults.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4caeef87f87a4bb7"}},{"code_sha256_prefix":"dbc9cac27d1c2130","entry":"format_as_iter","repo":"batuozt/gleam","repo_kind":"listed","path":"ss_recon/engine/dgl_trainer.py","file_url":"https://github.com/batuozt/gleam/blob/HEAD/ss_recon/engine/dgl_trainer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dbc9cac27d1c2130"}},{"code_sha256_prefix":"f8eba979963c16e9","entry":"get_files","repo":"batuozt/gleam","repo_kind":"listed","path":"datasets/format_fastmri.py","file_url":"https://github.com/batuozt/gleam/blob/HEAD/datasets/format_fastmri.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f8eba979963c16e9"}},{"code_sha256_prefix":"617c7d033c431ebe","entry":"ifftnc","repo":"batuozt/gleam","repo_kind":"listed","path":"datasets/utils/fftc.py","file_url":"https://github.com/batuozt/gleam/blob/HEAD/datasets/utils/fftc.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"617c7d033c431ebe"}},{"code_sha256_prefix":"d916b52fc1564a94","entry":"init_wandb_run","repo":"batuozt/gleam","repo_kind":"listed","path":"ss_recon/engine/defaults.py","file_url":"https://github.com/batuozt/gleam/blob/HEAD/ss_recon/engine/defaults.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d916b52fc1564a94"}},{"code_sha256_prefix":"7173d4cd30e0bb88","entry":"is_valid_file","repo":"batuozt/gleam","repo_kind":"listed","path":"datasets/format_fastmri.py","file_url":"https://github.com/batuozt/gleam/blob/HEAD/datasets/format_fastmri.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7173d4cd30e0bb88"}},{"code_sha256_prefix":"148b238d5ad66615","entry":"one_hot","repo":"eugenium/DGL","repo_kind":"official","path":"dni_comparisons/cifar_cnn_dni.py","file_url":"https://github.com/eugenium/DGL/blob/HEAD/dni_comparisons/cifar_cnn_dni.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"148b238d5ad66615"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}