{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/on-markov-chain-monte-carlo-methods-for-tall","title":"On Markov chain Monte Carlo methods for tall data","arxiv_id":"1505.02827","date":"2015-05-11","proceeding":null,"authors":["Rémi Bardenet","Arnaud Doucet","Chris Holmes"],"abstract":"Markov chain Monte Carlo methods are often deemed too computationally\nintensive to be of any practical use for big data applications, and in\nparticular for inference on datasets containing a large number $n$ of\nindividual data points, also known as tall datasets. In scenarios where data\nare assumed independent, various approaches to scale up the Metropolis-Hastings\nalgorithm in a Bayesian inference context have been recently proposed in\nmachine learning and computational statistics. These approaches can be grouped\ninto two categories: divide-and-conquer approaches and, subsampling-based\nalgorithms. The aims of this article are as follows. First, we present a\ncomprehensive review of the existing literature, commenting on the underlying\nassumptions and theoretical guarantees of each method. Second, by leveraging\nour understanding of these limitations, we propose an original\nsubsampling-based approach which samples from a distribution provably close to\nthe posterior distribution of interest, yet can require less than $O(n)$ data\npoint likelihood evaluations at each iteration for certain statistical models\nin favourable scenarios. Finally, we have only been able so far to propose\nsubsampling-based methods which display good performance in scenarios where the\nBernstein-von Mises approximation of the target posterior distribution is\nexcellent. It remains an open challenge to develop such methods in scenarios\nwhere the Bernstein-von Mises approximation is poor.","url_abs":"http://arxiv.org/abs/1505.02827v1","url_pdf":"http://arxiv.org/pdf/1505.02827v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"on-markov-chain-monte-carlo-methods-for-tall","repo_url":"https://github.com/rbardenet/2017-JMLR-MCMCForTallData","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"bayesian-inference","task_name":"Bayesian Inference"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1505.02827","atlas_url":"https://app.syntology.ai/?focus=1505.02827","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}