{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/random-reshuffling-simple-analysis-with-vast","title":"Random Reshuffling: Simple Analysis with Vast Improvements","arxiv_id":"2006.05988","date":"2020-06-10","proceeding":"NeurIPS 2020 12","authors":["Konstantin Mishchenko","Ahmed Khaled","Peter Richtárik"],"abstract":"Random Reshuffling (RR) is an algorithm for minimizing finite-sum functions that utilizes iterative gradient descent steps in conjunction with data reshuffling. Often contrasted with its sibling Stochastic Gradient Descent (SGD), RR is usually faster in practice and enjoys significant popularity in convex and non-convex optimization. The convergence rate of RR has attracted substantial attention recently and, for strongly convex and smooth functions, it was shown to converge faster than SGD if 1) the stepsize is small, 2) the gradients are bounded, and 3) the number of epochs is large. We remove these 3 assumptions, improve the dependence on the condition number from $\\kappa^2$ to $\\kappa$ (resp. from $\\kappa$ to $\\sqrt{\\kappa}$) and, in addition, show that RR has a different type of variance. We argue through theory and experiments that the new variance type gives an additional justification of the superior performance of RR. To go beyond strong convexity, we present several results for non-strongly convex and non-convex objectives. We show that in all cases, our theory improves upon existing literature. Finally, we prove fast convergence of the Shuffle-Once (SO) algorithm, which shuffles the data only once, at the beginning of the optimization process. Our theory for strongly-convex objectives tightly matches the known lower bounds for both RR and SO and substantiates the common practical heuristic of shuffling once or only a few times. As a byproduct of our analysis, we also get new results for the Incremental Gradient algorithm (IG), which does not shuffle the data at all.","url_abs":"https://arxiv.org/abs/2006.05988v3","url_pdf":"https://arxiv.org/pdf/2006.05988v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"random-reshuffling-simple-analysis-with-vast","repo_url":"https://github.com/konstmish/random_reshuffling","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[{"method_slug":"sgd","method_name":"SGD"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2006.05988","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2006.05988"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/konstmish/random_reshuffling","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_honours":1,"ran":1,"unverified":5},"by_repo_kind":{"official":{"samples":7,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"fbb68ed59acbdac7","entry":"logsig","repo":"konstmish/random_reshuffling","repo_kind":"official","path":"loss_functions/logistic_regression.py","file_url":"https://github.com/konstmish/random_reshuffling/blob/HEAD/loss_functions/logistic_regression.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fbb68ed59acbdac7"}},{"code_sha256_prefix":"fccd6875c293069d","entry":"relative_round","repo":"konstmish/random_reshuffling","repo_kind":"official","path":"utils.py","file_url":"https://github.com/konstmish/random_reshuffling/blob/HEAD/utils.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fccd6875c293069d"}},{"code_sha256_prefix":"93a358c02b10d447","entry":"get_dataset","repo":"konstmish/random_reshuffling","repo_kind":"official","path":"datasets/utils.py","file_url":"https://github.com/konstmish/random_reshuffling/blob/HEAD/datasets/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"93a358c02b10d447"}},{"code_sha256_prefix":"45ea328b237ba674","entry":"get_trace","repo":"konstmish/random_reshuffling","repo_kind":"official","path":"utils.py","file_url":"https://github.com/konstmish/random_reshuffling/blob/HEAD/utils.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"45ea328b237ba674"}},{"code_sha256_prefix":"8124a74030b255c0","entry":"safe_sparse_add","repo":"konstmish/random_reshuffling","repo_kind":"official","path":"loss_functions/utils.py","file_url":"https://github.com/konstmish/random_reshuffling/blob/HEAD/loss_functions/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8124a74030b255c0"}},{"code_sha256_prefix":"93448d7e10c42b22","entry":"safe_sparse_dot","repo":"konstmish/random_reshuffling","repo_kind":"official","path":"loss_functions/utils.py","file_url":"https://github.com/konstmish/random_reshuffling/blob/HEAD/loss_functions/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"93448d7e10c42b22"}},{"code_sha256_prefix":"2f70f473d9821025","entry":"safe_sparse_multiply","repo":"konstmish/random_reshuffling","repo_kind":"official","path":"loss_functions/utils.py","file_url":"https://github.com/konstmish/random_reshuffling/blob/HEAD/loss_functions/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2f70f473d9821025"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}