{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/barzilai-borwein-step-size-for-stochastic","title":"Barzilai-Borwein Step Size for Stochastic Gradient Descent","arxiv_id":"1605.04131","date":"2016-05-13","proceeding":"NeurIPS 2016 12","authors":["Conghui Tan","Shiqian Ma","Yu-Hong Dai","Yuqiu Qian"],"abstract":"One of the major issues in stochastic gradient descent (SGD) methods is how\nto choose an appropriate step size while running the algorithm. Since the\ntraditional line search technique does not apply for stochastic optimization\nalgorithms, the common practice in SGD is either to use a diminishing step\nsize, or to tune a fixed step size by hand, which can be time consuming in\npractice. In this paper, we propose to use the Barzilai-Borwein (BB) method to\nautomatically compute step sizes for SGD and its variant: stochastic variance\nreduced gradient (SVRG) method, which leads to two algorithms: SGD-BB and\nSVRG-BB. We prove that SVRG-BB converges linearly for strongly convex objective\nfunctions. As a by-product, we prove the linear convergence result of SVRG with\nOption I proposed in [10], whose convergence result is missing in the\nliterature. Numerical experiments on standard data sets show that the\nperformance of SGD-BB and SVRG-BB is comparable to and sometimes even better\nthan SGD and SVRG with best-tuned step sizes, and is superior to some advanced\nSGD variants.","url_abs":"http://arxiv.org/abs/1605.04131v2","url_pdf":"http://arxiv.org/pdf/1605.04131v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"barzilai-borwein-step-size-for-stochastic","repo_url":"https://github.com/SyneRBI/PETRIC-MaGeZ","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"stochastic-optimization","task_name":"Stochastic Optimization"}],"methods":[{"method_slug":"sgd","method_name":"SGD"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1605.04131","atlas_url":"https://app.syntology.ai/?focus=1605.04131","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1605.04131"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/SyneRBI/PETRIC-MaGeZ","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":7},"by_repo_kind":{"listed":{"samples":7,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"4a191a51c146b95d","entry":"load_interfile_image","repo":"SyneRBI/PETRIC-MaGeZ","repo_kind":"listed","path":"plot_data.py","file_url":"https://github.com/SyneRBI/PETRIC-MaGeZ/blob/HEAD/plot_data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4a191a51c146b95d"}},{"code_sha256_prefix":"2587457df9ca3fe1","entry":"load_interfile_image","repo":"SyneRBI/PETRIC-MaGeZ","repo_kind":"listed","path":"simulation_src/fig7_fig_8_challenge.py","file_url":"https://github.com/SyneRBI/PETRIC-MaGeZ/blob/HEAD/simulation_src/fig7_fig_8_challenge.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2587457df9ca3fe1"}},{"code_sha256_prefix":"31f80f211d49b55e","entry":"neighbor_difference_and_sum","repo":"SyneRBI/PETRIC-MaGeZ","repo_kind":"listed","path":"rdp.py","file_url":"https://github.com/SyneRBI/PETRIC-MaGeZ/blob/HEAD/rdp.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"31f80f211d49b55e"}},{"code_sha256_prefix":"e7134a80c918c0a7","entry":"neighbor_offsets","repo":"SyneRBI/PETRIC-MaGeZ","repo_kind":"listed","path":"rdp.py","file_url":"https://github.com/SyneRBI/PETRIC-MaGeZ/blob/HEAD/rdp.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e7134a80c918c0a7"}},{"code_sha256_prefix":"82e3d53f70395607","entry":"neighbor_product","repo":"SyneRBI/PETRIC-MaGeZ","repo_kind":"listed","path":"rdp.py","file_url":"https://github.com/SyneRBI/PETRIC-MaGeZ/blob/HEAD/rdp.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"82e3d53f70395607"}},{"code_sha256_prefix":"a0ebd0be7d6e9f8f","entry":"parse_interfile_header","repo":"SyneRBI/PETRIC-MaGeZ","repo_kind":"listed","path":"plot_data.py","file_url":"https://github.com/SyneRBI/PETRIC-MaGeZ/blob/HEAD/plot_data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a0ebd0be7d6e9f8f"}},{"code_sha256_prefix":"7962b724006daa74","entry":"scalars","repo":"SyneRBI/PETRIC-MaGeZ","repo_kind":"listed","path":"simulation_src/fig7_fig_8_challenge.py","file_url":"https://github.com/SyneRBI/PETRIC-MaGeZ/blob/HEAD/simulation_src/fig7_fig_8_challenge.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7962b724006daa74"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}