{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-biologically-inspired-visual-working-memory","title":"A Biologically Inspired Visual Working Memory for Deep Networks","arxiv_id":"1901.03665","date":"2019-01-09","proceeding":"ICLR 2019 5","authors":["Ethan Harris","Mahesan Niranjan","Jonathon Hare"],"abstract":"The ability to look multiple times through a series of pose-adjusted glimpses\nis fundamental to human vision. This critical faculty allows us to understand\nhighly complex visual scenes. Short term memory plays an integral role in\naggregating the information obtained from these glimpses and informing our\ninterpretation of the scene. Computational models have attempted to address\nglimpsing and visual attention but have failed to incorporate the notion of\nmemory. We introduce a novel, biologically inspired visual working memory\narchitecture that we term the Hebb-Rosenblatt memory. We subsequently introduce\na fully differentiable Short Term Attentive Working Memory model (STAWM) which\nuses transformational attention to learn a memory over each image it sees. The\nstate of our Hebb-Rosenblatt memory is embedded in STAWM as the weights space\nof a layer. By projecting different queries through this layer we can obtain\ngoal-oriented latent representations for tasks including classification and\nvisual reconstruction. Our model obtains highly competitive classification\nperformance on MNIST and CIFAR-10. As demonstrated through the CelebA dataset,\nto perform reconstruction the model learns to make a sequence of updates to a\ncanvas which constitute a parts-based representation. Classification with the\nself supervised representation obtained from MNIST is shown to be in line with\nthe state of the art models (none of which use a visual attention mechanism).\nFinally, we show that STAWM can be trained under the dual constraints of\nclassification and reconstruction to provide an interpretable visual sketchpad\nwhich helps open the 'black-box' of deep learning.","url_abs":"http://arxiv.org/abs/1901.03665v1","url_pdf":"http://arxiv.org/pdf/1901.03665v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-biologically-inspired-visual-working-memory","repo_url":"https://github.com/ethanwharris/STAWM","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"a-biologically-inspired-visual-working-memory","repo_url":"https://github.com/iclr2019-anon/STAWM","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"classification-1","task_name":"Classification"},{"task_slug":"classification","task_name":"General Classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}