{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/dependent-relevance-determination-for-smooth","title":"Dependent relevance determination for smooth and structured sparse regression","arxiv_id":"1711.10058","date":"2017-11-28","proceeding":null,"authors":["Anqi Wu","Oluwasanmi Koyejo","Jonathan W. Pillow"],"abstract":"In many problem settings, parameter vectors are not merely sparse but\ndependent in such a way that non-zero coefficients tend to cluster together. We\nrefer to this form of dependency as \"region sparsity.\" Classical sparse\nregression methods, such as the lasso and automatic relevance determination\n(ARD), which model parameters as independent a priori, and therefore do not\nexploit such dependencies. Here we introduce a hierarchical model for smooth,\nregion-sparse weight vectors and tensors in a linear regression setting. Our\napproach represents a hierarchical extension of the relevance determination\nframework, where we add a transformed Gaussian process to model the\ndependencies between the prior variances of regression weights. We combine this\nwith a structured model of the prior variances of Fourier coefficients, which\neliminates unnecessary high frequencies. The resulting prior encourages weights\nto be region-sparse in two different bases simultaneously. We develop Laplace\napproximation and Monte Carlo Markov Chain (MCMC) sampling to provide efficient\ninference for the posterior. Furthermore, a two-stage convex relaxation of the\nLaplace approximation approach is also provided to relax the inevitable\nnon-convexity during the optimization. We finally show substantial improvements\nover comparable methods for both simulated and real datasets from brain\nimaging.","url_abs":"http://arxiv.org/abs/1711.10058v3","url_pdf":"http://arxiv.org/pdf/1711.10058v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"dependent-relevance-determination-for-smooth","repo_url":"https://github.com/waq1129/DRD","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"regression-1","task_name":"regression"}],"methods":[{"method_slug":"gaussian-process","method_name":"Gaussian Process"},{"method_slug":"linear-regression","method_name":"Linear Regression"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1711.10058","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}