{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-spectral-approach-to-gradient-estimation","title":"A Spectral Approach to Gradient Estimation for Implicit Distributions","arxiv_id":"1806.02925","date":"2018-06-07","proceeding":"ICML 2018 7","authors":["Jiaxin Shi","Shengyang Sun","Jun Zhu"],"abstract":"Recently there have been increasing interests in learning and inference with\nimplicit distributions (i.e., distributions without tractable densities). To\nthis end, we develop a gradient estimator for implicit distributions based on\nStein's identity and a spectral decomposition of kernel operators, where the\neigenfunctions are approximated by the Nystr\\\"om method. Unlike the previous\nworks that only provide estimates at the sample points, our approach directly\nestimates the gradient function, thus allows for a simple and principled\nout-of-sample extension. We provide theoretical results on the error bound of\nthe estimator and discuss the bias-variance tradeoff in practice. The\neffectiveness of our method is demonstrated by applications to gradient-free\nHamiltonian Monte Carlo and variational inference with implicit distributions.\nFinally, we discuss the intuition behind the estimator by drawing connections\nbetween the Nystr\\\"om method and kernel PCA, which indicates that the estimator\ncan automatically adapt to the geometry of the underlying distribution.","url_abs":"http://arxiv.org/abs/1806.02925v1","url_pdf":"http://arxiv.org/pdf/1806.02925v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-spectral-approach-to-gradient-estimation","repo_url":"https://github.com/thjashin/spectral-stein-grad","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"a-spectral-approach-to-gradient-estimation","repo_url":"https://github.com/AntixK/Spectral-Stein-Gradient","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"a-spectral-approach-to-gradient-estimation","repo_url":"https://github.com/nonconvexopt/pytorch-SSGE","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"variational-inference","task_name":"Variational Inference"}],"methods":[{"method_slug":"pca","method_name":"PCA"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=1806.02925","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}