{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/on-multi-layer-basis-pursuit-efficient","title":"On Multi-Layer Basis Pursuit, Efficient Algorithms and Convolutional Neural Networks","arxiv_id":"1806.00701","date":"2018-06-02","proceeding":null,"authors":["Jeremias Sulam","Aviad Aberdam","Amir Beck","Michael Elad"],"abstract":"Parsimonious representations are ubiquitous in modeling and processing\ninformation. Motivated by the recent Multi-Layer Convolutional Sparse Coding\n(ML-CSC) model, we herein generalize the traditional Basis Pursuit problem to a\nmulti-layer setting, introducing similar sparse enforcing penalties at\ndifferent representation layers in a symbiotic relation between synthesis and\nanalysis sparse priors. We explore different iterative methods to solve this\nnew problem in practice, and we propose a new Multi-Layer Iterative Soft\nThresholding Algorithm (ML-ISTA), as well as a fast version (ML-FISTA). We show\nthat these nested first order algorithms converge, in the sense that the\nfunction value of near-fixed points can get arbitrarily close to the solution\nof the original problem.\n  We further show how these algorithms effectively implement particular\nrecurrent convolutional neural networks (CNNs) that generalize feed-forward\nones without introducing any parameters. We present and analyze different\narchitectures resulting unfolding the iterations of the proposed pursuit\nalgorithms, including a new Learned ML-ISTA, providing a principled way to\nconstruct deep recurrent CNNs. Unlike other similar constructions, these\narchitectures unfold a global pursuit holistically for the entire network. We\ndemonstrate the emerging constructions in a supervised learning setting,\nconsistently improving the performance of classical CNNs while maintaining the\nnumber of parameters constant.","url_abs":"http://arxiv.org/abs/1806.00701v5","url_pdf":"http://arxiv.org/pdf/1806.00701v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"on-multi-layer-basis-pursuit-efficient","repo_url":"https://github.com/Sulam-Group/ml-ista","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"on-multi-layer-basis-pursuit-efficient","repo_url":"https://github.com/jsulam/ml-ista","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1806.00701","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}