{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/on-better-training-the-infinite-restricted","title":"On better training the infinite restricted Boltzmann machines","arxiv_id":"1709.03239","date":"2017-09-11","proceeding":null,"authors":["Xuan Peng","Xunzhang Gao","Xiang Li"],"abstract":"The infinite restricted Boltzmann machine (iRBM) is an extension of the\nclassic RBM. It enjoys a good property of automatically deciding the size of\nthe hidden layer according to specific training data. With sufficient training,\nthe iRBM can achieve a competitive performance with that of the classic RBM.\nHowever, the convergence of learning the iRBM is slow, due to the fact that the\niRBM is sensitive to the ordering of its hidden units, the learned filters\nchange slowly from the left-most hidden unit to right. To break this dependency\nbetween neighboring hidden units and speed up the convergence of training, a\nnovel training strategy is proposed. The key idea of the proposed training\nstrategy is randomly regrouping the hidden units before each gradient descent\nstep. Potentially, a mixing of infinite many iRBMs with different permutations\nof the hidden units can be achieved by this learning method, which has a\nsimilar effect of preventing the model from over-fitting as the dropout. The\noriginal iRBM is also modified to be capable of carrying out discriminative\ntraining. To evaluate the impact of our method on convergence speed of learning\nand the model's generalization ability, several experiments have been performed\non the binarized MNIST and CalTech101 Silhouettes datasets. Experimental\nresults indicate that the proposed training strategy can greatly accelerate\nlearning and enhance generalization ability of iRBMs.","url_abs":"http://arxiv.org/abs/1709.03239v2","url_pdf":"http://arxiv.org/pdf/1709.03239v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"on-better-training-the-infinite-restricted","repo_url":"https://github.com/Boltzxuann/RP-iRBM","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[],"methods":[{"method_slug":"restricted-boltzmann-machine","method_name":"Restricted Boltzmann Machine"},{"method_slug":"speed","method_name":"SPEED"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}