{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/optimal-deep-neural-networks-for-sparse","title":"Optimal deep neural networks for sparse recovery via Laplace techniques","arxiv_id":"1709.01112","date":"2017-09-04","proceeding":null,"authors":["Steffen Limmer","Slawomir Stanczak"],"abstract":"This paper introduces Laplace techniques for designing a neural network, with\nthe goal of estimating simplex-constraint sparse vectors from compressed\nmeasurements. To this end, we recast the problem of MMSE estimation (w.r.t. a\npre-defined uniform input distribution) as the problem of computing the\ncentroid of some polytope that results from the intersection of the simplex and\nan affine subspace determined by the measurements. Owing to the specific\nstructure, it is shown that the centroid can be computed analytically by\nextending a recent result that facilitates the volume computation of polytopes\nvia Laplace transformations. A main insight of this paper is that the desired\nvolume and centroid computations can be performed by a classical deep neural\nnetwork comprising threshold functions, rectified linear (ReLU) and rectified\npolynomial (ReP) activation functions. The proposed construction of a deep\nneural network for sparse recovery is completely analytic so that\ntime-consuming training procedures are not necessary. Furthermore, we show that\nthe number of layers in our construction is equal to the number of measurements\nwhich might enable novel low-latency sparse recovery algorithms for a larger\nclass of signals than that assumed in this paper. To assess the applicability\nof the proposed uniform input distribution, we showcase the recovery\nperformance on samples that are soft-classification vectors generated by two\nstandard datasets. As both volume and centroid computation are known to be\ncomputationally hard, the network width grows exponentially in the worst-case.\nIt can be, however, decreased by inducing sparse connectivity in the neural\nnetwork via a well-suited basis of the affine subspace. Finally, the presented\nanalytical construction may serve as a viable initialization to be further\noptimized and trained using particular input datasets at hand.","url_abs":"http://arxiv.org/abs/1709.01112v2","url_pdf":"http://arxiv.org/pdf/1709.01112v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"optimal-deep-neural-networks-for-sparse","repo_url":"https://github.com/stli/CentNet","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}