{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/tensor-monte-carlo-particle-methods-for-the","title":"Tensor Monte Carlo: particle methods for the GPU era","arxiv_id":"1806.08593","date":"2018-06-22","proceeding":"NeurIPS 2019 12","authors":["Laurence Aitchison"],"abstract":"Multi-sample, importance-weighted variational autoencoders (IWAE) give\ntighter bounds and more accurate uncertainty estimates than variational\nautoencoders (VAE) trained with a standard single-sample objective. However,\nIWAEs scale poorly: as the latent dimensionality grows, they require\nexponentially many samples to retain the benefits of importance weighting.\nWhile sequential Monte-Carlo (SMC) can address this problem, it is\nprohibitively slow because the resampling step imposes sequential structure\nwhich cannot be parallelised, and moreover, resampling is non-differentiable\nwhich is problematic when learning approximate posteriors. To address these\nissues, we developed tensor Monte-Carlo (TMC) which gives exponentially many\nimportance samples by separately drawing $K$ samples for each of the $n$ latent\nvariables, then averaging over all $K^n$ possible combinations. While the sum\nover exponentially many terms might seem to be intractable, in many cases it\ncan be computed efficiently as a series of tensor inner-products. We show that\nTMC is superior to IWAE on a generative model with multiple stochastic layers\ntrained on the MNIST handwritten digit database, and we show that TMC can be\ncombined with standard variance reduction techniques.","url_abs":"http://arxiv.org/abs/1806.08593v3","url_pdf":"http://arxiv.org/pdf/1806.08593v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"tensor-monte-carlo-particle-methods-for-the","repo_url":"https://github.com/anonymous-78913/tmc-anon","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":null,"task_name":"GPU"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1806.08593","atlas_url":"https://app.syntology.ai/?focus=1806.08593","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}