{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/same-but-different-fast-and-high-quality","title":"SAME but Different: Fast and High-Quality Gibbs Parameter Estimation","arxiv_id":"1409.5402","date":"2014-09-18","proceeding":null,"authors":["Huasha Zhao","Biye Jiang","John Canny"],"abstract":"Gibbs sampling is a workhorse for Bayesian inference but has several\nlimitations when used for parameter estimation, and is often much slower than\nnon-sampling inference methods. SAME (State Augmentation for Marginal\nEstimation) \\cite{Doucet99,Doucet02} is an approach to MAP parameter estimation\nwhich gives improved parameter estimates over direct Gibbs sampling. SAME can\nbe viewed as cooling the posterior parameter distribution and allows annealed\nsearch for the MAP parameters, often yielding very high quality (lower loss)\nestimates. But it does so at the expense of additional samples per iteration\nand generally slower performance. On the other hand, SAME dramatically\nincreases the parallelism in the sampling schedule, and is an excellent match\nfor modern (SIMD) hardware. In this paper we explore the application of SAME to\ngraphical model inference on modern hardware. We show that combining SAME with\nfactored sample representation (or approximation) gives throughput competitive\nwith the fastest symbolic methods, but with potentially better quality. We\ndescribe experiments on Latent Dirichlet Allocation, achieving speeds similar\nto the fastest reported methods (online Variational Bayes) and lower\ncross-validated loss than other LDA implementations. The method is simple to\nimplement and should be applicable to many other models.","url_abs":"http://arxiv.org/abs/1409.5402v1","url_pdf":"http://arxiv.org/pdf/1409.5402v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"same-but-different-fast-and-high-quality","repo_url":"https://github.com/BIDData/BIDMach","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null},{"paper_slug":"same-but-different-fast-and-high-quality","repo_url":"https://github.com/danrugeles/Heron","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"bayesian-inference","task_name":"Bayesian Inference"},{"task_slug":"high","task_name":"Vocal Bursts Intensity Prediction"},{"task_slug":"parameter-estimation","task_name":"parameter estimation"}],"methods":[{"method_slug":"lda","method_name":"LDA"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}