{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/bidirectional-helmholtz-machines","title":"Bidirectional Helmholtz Machines","arxiv_id":"1506.03877","date":"2015-06-12","proceeding":null,"authors":["Jorg Bornschein","Samira Shabanian","Asja Fischer","Yoshua Bengio"],"abstract":"Efficient unsupervised training and inference in deep generative models\nremains a challenging problem. One basic approach, called Helmholtz machine,\ninvolves training a top-down directed generative model together with a\nbottom-up auxiliary model used for approximate inference. Recent results\nindicate that better generative models can be obtained with better approximate\ninference procedures. Instead of improving the inference procedure, we here\npropose a new model which guarantees that the top-down and bottom-up\ndistributions can efficiently invert each other. We achieve this by\ninterpreting both the top-down and the bottom-up directed models as approximate\ninference distributions and by defining the model distribution to be the\ngeometric mean of these two. We present a lower-bound for the likelihood of\nthis model and we show that optimizing this bound regularizes the model so that\nthe Bhattacharyya distance between the bottom-up and top-down approximate\ndistributions is minimized. This approach results in state of the art\ngenerative models which prefer significantly deeper architectures while it\nallows for orders of magnitude more efficient approximate inference.","url_abs":"http://arxiv.org/abs/1506.03877v5","url_pdf":"http://arxiv.org/pdf/1506.03877v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"bidirectional-helmholtz-machines","repo_url":"https://github.com/jbornschein/bihm","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1506.03877","atlas_url":"https://app.syntology.ai/?focus=1506.03877","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}