{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/understanding-minimum-probability-flow-for","title":"Understanding Minimum Probability Flow for RBMs Under Various Kinds of Dynamics","arxiv_id":"1412.6617","date":"2014-12-20","proceeding":null,"authors":["Daniel Jiwoong Im","Ethan Buchman","Graham W. Taylor"],"abstract":"Energy-based models are popular in machine learning due to the elegance of\ntheir formulation and their relationship to statistical physics. Among these,\nthe Restricted Boltzmann Machine (RBM), and its staple training algorithm\ncontrastive divergence (CD), have been the prototype for some recent\nadvancements in the unsupervised training of deep neural networks. However, CD\nhas limited theoretical motivation, and can in some cases produce undesirable\nbehavior. Here, we investigate the performance of Minimum Probability Flow\n(MPF) learning for training RBMs. Unlike CD, with its focus on approximating an\nintractable partition function via Gibbs sampling, MPF proposes a tractable,\nconsistent, objective function defined in terms of a Taylor expansion of the KL\ndivergence with respect to sampling dynamics. Here we propose a more general\nform for the sampling dynamics in MPF, and explore the consequences of\ndifferent choices for these dynamics for training RBMs. Experimental results\nshow MPF outperforming CD for various RBM configurations.","url_abs":"http://arxiv.org/abs/1412.6617v6","url_pdf":"http://arxiv.org/pdf/1412.6617v6.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"understanding-minimum-probability-flow-for","repo_url":"https://github.com/jiwoongim/minimum_probability_flow_learning","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[],"methods":[{"method_slug":"restricted-boltzmann-machine","method_name":"Restricted Boltzmann Machine"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}