{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/beyond-backprop-online-alternating","title":"Beyond Backprop: Online Alternating Minimization with Auxiliary Variables","arxiv_id":"1806.09077","date":"2018-06-24","proceeding":null,"authors":["Anna Choromanska","Benjamin Cowen","Sadhana Kumaravel","Ronny Luss","Mattia Rigotti","Irina Rish","Brian Kingsbury","Paolo DiAchille","Viatcheslav Gurev","Ravi Tejwani","Djallel Bouneffouf"],"abstract":"Despite significant recent advances in deep neural networks, training them remains a challenge due to the highly non-convex nature of the objective function. State-of-the-art methods rely on error backpropagation, which suffers from several well-known issues, such as vanishing and exploding gradients, inability to handle non-differentiable nonlinearities and to parallelize weight-updates across layers, and biological implausibility. These limitations continue to motivate exploration of alternative training algorithms, including several recently proposed auxiliary-variable methods which break the complex nested objective function into local subproblems. However, those techniques are mainly offline (batch), which limits their applicability to extremely large datasets, as well as to online, continual or reinforcement learning. The main contribution of our work is a novel online (stochastic/mini-batch) alternating minimization (AM) approach for training deep neural networks, together with the first theoretical convergence guarantees for AM in stochastic settings and promising empirical results on a variety of architectures and datasets.","url_abs":"https://arxiv.org/abs/1806.09077v4","url_pdf":"https://arxiv.org/pdf/1806.09077v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"beyond-backprop-online-alternating","repo_url":"https://github.com/IBM/online-alt-min","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"}],"methods":[{"method_slug":"am","method_name":"AM"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1806.09077","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1806.09077"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/IBM/online-alt-min","reach":null}],"summary":{"ran_draft_wrong":1,"ran_honours":1,"ran_fixture":1,"unverified":1},"by_repo_kind":{"official":{"samples":4,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"553746773aa19050","entry":"compute_codes_loss","repo":"IBM/online-alt-min","repo_kind":"official","path":"altmin.py","file_url":"https://github.com/IBM/online-alt-min/blob/HEAD/altmin.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"553746773aa19050"}},{"code_sha256_prefix":"f8aaf93035e8a3d1","entry":"get_stat_data_files","repo":"IBM/online-alt-min","repo_kind":"official","path":"plot_results.py","file_url":"https://github.com/IBM/online-alt-min/blob/HEAD/plot_results.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f8aaf93035e8a3d1"}},{"code_sha256_prefix":"a1d8ad42b49ed98d","entry":"update_memory","repo":"IBM/online-alt-min","repo_kind":"official","path":"altmin.py","file_url":"https://github.com/IBM/online-alt-min/blob/HEAD/altmin.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a1d8ad42b49ed98d"}},{"code_sha256_prefix":"9312660deb4b5ab2","entry":"BCD","repo":"IBM/online-alt-min","repo_kind":"official","path":"altmin.py","file_url":"https://github.com/IBM/online-alt-min/blob/HEAD/altmin.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9312660deb4b5ab2"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}