{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/understanding-the-learned-iterative-soft","title":"Understanding the Learned Iterative Soft Thresholding Algorithm with matrix factorization","arxiv_id":"1706.01338","date":"2017-06-02","proceeding":null,"authors":["Thomas Moreau","Joan Bruna"],"abstract":"Sparse coding is a core building block in many data analysis and machine\nlearning pipelines. Typically it is solved by relying on generic optimization\ntechniques, such as the Iterative Soft Thresholding Algorithm and its\naccelerated version (ISTA, FISTA). These methods are optimal in the class of\nfirst-order methods for non-smooth, convex functions. However, they do not\nexploit the particular structure of the problem at hand nor the input data\ndistribution. An acceleration using neural networks, coined LISTA, was proposed\nin Gregor and Le Cun (2010), which showed empirically that one could achieve\nhigh quality estimates with few iterations by modifying the parameters of the\nproximal splitting appropriately.\n  In this paper we study the reasons for such acceleration. Our mathematical\nanalysis reveals that it is related to a specific matrix factorization of the\nGram kernel of the dictionary, which attempts to nearly diagonalise the kernel\nwith a basis that produces a small perturbation of the $\\ell_1$ ball. When this\nfactorization succeeds, we prove that the resulting splitting algorithm enjoys\nan improved convergence bound with respect to the non-adaptive version.\nMoreover, our analysis also shows that conditions for acceleration occur mostly\nat the beginning of the iterative process, consistent with numerical\nexperiments. We further validate our analysis by showing that on dictionaries\nwhere this factorization does not exist, adaptive acceleration fails.","url_abs":"http://arxiv.org/abs/1706.01338v1","url_pdf":"http://arxiv.org/pdf/1706.01338v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"understanding-the-learned-iterative-soft","repo_url":"https://github.com/tomMoral/AdaptiveOptim","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1706.01338","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1706.01338"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tomMoral/AdaptiveOptim","reach":null}],"summary":{"unverified":2},"by_repo_kind":{"official":{"samples":2,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"d8eb74aaf0c1af20","entry":"get_problem","repo":"tomMoral/AdaptiveOptim","repo_kind":"official","path":"NIPS_figures.py","file_url":"https://github.com/tomMoral/AdaptiveOptim/blob/HEAD/NIPS_figures.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d8eb74aaf0c1af20"}},{"code_sha256_prefix":"9e4856c10dbdfdea","entry":"parse_runfile","repo":"tomMoral/AdaptiveOptim","repo_kind":"official","path":"NIPS_figures.py","file_url":"https://github.com/tomMoral/AdaptiveOptim/blob/HEAD/NIPS_figures.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9e4856c10dbdfdea"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}