{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/minimizing-finite-sums-with-the-stochastic","title":"Minimizing Finite Sums with the Stochastic Average Gradient","arxiv_id":"1309.2388","date":"2013-09-10","proceeding":null,"authors":["Mark Schmidt","Nicolas Le Roux","Francis Bach"],"abstract":"We propose the stochastic average gradient (SAG) method for optimizing the\nsum of a finite number of smooth convex functions. Like stochastic gradient\n(SG) methods, the SAG method's iteration cost is independent of the number of\nterms in the sum. However, by incorporating a memory of previous gradient\nvalues the SAG method achieves a faster convergence rate than black-box SG\nmethods. The convergence rate is improved from O(1/k^{1/2}) to O(1/k) in\ngeneral, and when the sum is strongly-convex the convergence rate is improved\nfrom the sub-linear O(1/k) to a linear convergence rate of the form O(p^k) for\np \\textless{} 1. Further, in many cases the convergence rate of the new method\nis also faster than black-box deterministic gradient methods, in terms of the\nnumber of gradient evaluations. Numerical experiments indicate that the new\nalgorithm often dramatically outperforms existing SG and deterministic gradient\nmethods, and that the performance may be further improved through the use of\nnon-uniform sampling strategies.","url_abs":"http://arxiv.org/abs/1309.2388v2","url_pdf":"http://arxiv.org/pdf/1309.2388v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"minimizing-finite-sums-with-the-stochastic","repo_url":"https://github.com/SimonDele/School-projects","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"minimizing-finite-sums-with-the-stochastic","repo_url":"https://github.com/nathansiae/Stochastic-Average-Newton","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1309.2388","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1309.2388"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/SimonDele/School-projects","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/nathansiae/Stochastic-Average-Newton","reach":null}],"summary":{"ran_draft_wrong":3},"by_repo_kind":{"listed":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"cee8ca9f51ef6db5","entry":"sag","repo":"nathansiae/Stochastic-Average-Newton","repo_kind":"listed","path":"src/algorithms.py","file_url":"https://github.com/nathansiae/Stochastic-Average-Newton/blob/HEAD/src/algorithms.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cee8ca9f51ef6db5"}},{"code_sha256_prefix":"8705a4be8f7bfb4e","entry":"san","repo":"nathansiae/Stochastic-Average-Newton","repo_kind":"listed","path":"src/algorithms.py","file_url":"https://github.com/nathansiae/Stochastic-Average-Newton/blob/HEAD/src/algorithms.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"none","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8705a4be8f7bfb4e"}},{"code_sha256_prefix":"7b7ff2053081afac","entry":"sana","repo":"nathansiae/Stochastic-Average-Newton","repo_kind":"listed","path":"src/algorithms.py","file_url":"https://github.com/nathansiae/Stochastic-Average-Newton/blob/HEAD/src/algorithms.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"none","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7b7ff2053081afac"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}