{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/stochastic-chebyshev-gradient-descent-for","title":"Stochastic Chebyshev Gradient Descent for Spectral Optimization","arxiv_id":"1802.06355","date":"2018-02-18","proceeding":"NeurIPS 2018 12","authors":["Insu Han","Haim Avron","Jinwoo Shin"],"abstract":"A large class of machine learning techniques requires the solution of\noptimization problems involving spectral functions of parametric matrices, e.g.\nlog-determinant and nuclear norm. Unfortunately, computing the gradient of a\nspectral function is generally of cubic complexity, as such gradient descent\nmethods are rather expensive for optimizing objectives involving the spectral\nfunction. Thus, one naturally turns to stochastic gradient methods in hope that\nthey will provide a way to reduce or altogether avoid the computation of full\ngradients. However, here a new challenge appears: there is no straightforward\nway to compute unbiased stochastic gradients for spectral functions. In this\npaper, we develop unbiased stochastic gradients for spectral-sums, an important\nsubclass of spectral functions. Our unbiased stochastic gradients are based on\ncombining randomized trace estimators with stochastic truncation of the\nChebyshev expansions. A careful design of the truncation distribution allows us\nto offer distributions that are variance-optimal, which is crucial for fast and\nstable convergence of stochastic gradient methods. We further leverage our\nproposed stochastic gradients to devise stochastic methods for objective\nfunctions involving spectral-sums, and rigorously analyze their convergence\nrate. The utility of our methods is demonstrated in numerical experiments.","url_abs":"http://arxiv.org/abs/1802.06355v3","url_pdf":"http://arxiv.org/pdf/1802.06355v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"stochastic-chebyshev-gradient-descent-for","repo_url":"https://github.com/EiffL/SpectralFlow","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1802.06355","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1802.06355"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/EiffL/SpectralFlow","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":4},"by_repo_kind":{"listed":{"samples":4,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"faf705e7e2809add","entry":"ChebyshevCoefficients","repo":"EiffL/SpectralFlow","repo_kind":"listed","path":"spectralflow/chebyshev.py","file_url":"https://github.com/EiffL/SpectralFlow/blob/HEAD/spectralflow/chebyshev.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"faf705e7e2809add"}},{"code_sha256_prefix":"ce3b1fc44c04f170","entry":"chebyshev_logdet","repo":"EiffL/SpectralFlow","repo_kind":"listed","path":"spectralflow/logdet.py","file_url":"https://github.com/EiffL/SpectralFlow/blob/HEAD/spectralflow/logdet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ce3b1fc44c04f170"}},{"code_sha256_prefix":"3dc17597cc986fac","entry":"chebyshev_spectral_sum","repo":"EiffL/SpectralFlow","repo_kind":"listed","path":"spectralflow/logdet.py","file_url":"https://github.com/EiffL/SpectralFlow/blob/HEAD/spectralflow/logdet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3dc17597cc986fac"}},{"code_sha256_prefix":"e82aa7676d3d5d9c","entry":"eigen_max","repo":"EiffL/SpectralFlow","repo_kind":"listed","path":"spectralflow/ops.py","file_url":"https://github.com/EiffL/SpectralFlow/blob/HEAD/spectralflow/ops.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e82aa7676d3d5d9c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}