{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/variational-memory-addressing-in-generative","title":"Variational Memory Addressing in Generative Models","arxiv_id":"1709.07116","date":"2017-09-21","proceeding":"NeurIPS 2017 12","authors":["Jörg Bornschein","andriy mnih","Daniel Zoran","Danilo J. Rezende"],"abstract":"Aiming to augment generative models with external memory, we interpret the\noutput of a memory module with stochastic addressing as a conditional mixture\ndistribution, where a read operation corresponds to sampling a discrete memory\naddress and retrieving the corresponding content from memory. This perspective\nallows us to apply variational inference to memory addressing, which enables\neffective training of the memory module by using the target information to\nguide memory lookups. Stochastic addressing is particularly well-suited for\ngenerative models as it naturally encourages multimodality which is a prominent\naspect of most high-dimensional datasets. Treating the chosen address as a\nlatent variable also allows us to quantify the amount of information gained\nwith a memory lookup and measure the contribution of the memory module to the\ngenerative process. To illustrate the advantages of this approach we\nincorporate it into a variational autoencoder and apply the resulting model to\nthe task of generative few-shot learning. The intuition behind this\narchitecture is that the memory module can pick a relevant template from memory\nand the continuous part of the model can concentrate on modeling remaining\nvariations. We demonstrate empirically that our model is able to identify and\naccess the relevant memory contents even with hundreds of unseen Omniglot\ncharacters in memory","url_abs":"http://arxiv.org/abs/1709.07116v1","url_pdf":"http://arxiv.org/pdf/1709.07116v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"variational-memory-addressing-in-generative","repo_url":"https://github.com/artemZholus/variational_memory_addressing","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"few-shot-learning","task_name":"Few-Shot Learning"},{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"variational-inference","task_name":"Variational Inference"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1709.07116","atlas_url":"https://app.syntology.ai/?focus=1709.07116","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}