{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/horseshoe-regularization-for-feature-subset","title":"Horseshoe Regularization for Feature Subset Selection","arxiv_id":"1702.07400","date":"2017-02-23","proceeding":null,"authors":["Anindya Bhadra","Jyotishka Datta","Nicholas G. Polson","Brandon Willard"],"abstract":"Feature subset selection arises in many high-dimensional applications of\nstatistics, such as compressed sensing and genomics. The $\\ell_0$ penalty is\nideal for this task, the caveat being it requires the NP-hard combinatorial\nevaluation of all models. A recent area of considerable interest is to develop\nefficient algorithms to fit models with a non-convex $\\ell_\\gamma$ penalty for\n$\\gamma\\in (0,1)$, which results in sparser models than the convex $\\ell_1$ or\nlasso penalty, but is harder to fit. We propose an alternative, termed the\nhorseshoe regularization penalty for feature subset selection, and demonstrate\nits theoretical and computational advantages. The distinguishing feature from\nexisting non-convex optimization approaches is a full probabilistic\nrepresentation of the penalty as the negative of the logarithm of a suitable\nprior, which in turn enables efficient expectation-maximization and local\nlinear approximation algorithms for optimization and MCMC for uncertainty\nquantification. In synthetic and real data, the resulting algorithms provide\nbetter statistical performance, and the computation requires a fraction of time\nof state-of-the-art non-convex solvers.","url_abs":"http://arxiv.org/abs/1702.07400v2","url_pdf":"http://arxiv.org/pdf/1702.07400v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"horseshoe-regularization-for-feature-subset","repo_url":"https://github.com/ysfoo/crn-inference","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"uncertainty-quantification","task_name":"Uncertainty Quantification"},{"task_slug":"compressed-sensing","task_name":"compressed sensing"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}