{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/using-fast-weights-to-attend-to-the-recent","title":"Using Fast Weights to Attend to the Recent Past","arxiv_id":"1610.06258","date":"2016-10-20","proceeding":"NeurIPS 2016 12","authors":["Jimmy Ba","Geoffrey Hinton","Volodymyr Mnih","Joel Z. Leibo","Catalin Ionescu"],"abstract":"Until recently, research on artificial neural networks was largely restricted\nto systems with only two types of variable: Neural activities that represent\nthe current or recent input and weights that learn to capture regularities\namong inputs, outputs and payoffs. There is no good reason for this\nrestriction. Synapses have dynamics at many different time-scales and this\nsuggests that artificial neural networks might benefit from variables that\nchange slower than activities but much faster than the standard weights. These\n\"fast weights\" can be used to store temporary memories of the recent past and\nthey provide a neurally plausible way of implementing the type of attention to\nthe past that has recently proved very helpful in sequence-to-sequence models.\nBy using fast weights we can avoid the need to store copies of neural activity\npatterns.","url_abs":"http://arxiv.org/abs/1610.06258v3","url_pdf":"http://arxiv.org/pdf/1610.06258v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"using-fast-weights-to-attend-to-the-recent","repo_url":"https://github.com/GokuMohandas/fast-weights","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}},{"paper_slug":"using-fast-weights-to-attend-to-the-recent","repo_url":"https://github.com/jiamings/fast-weights","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"using-fast-weights-to-attend-to-the-recent","repo_url":"https://github.com/mwufi/fast-weights","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1610.06258","atlas_url":"https://app.syntology.ai/?focus=1610.06258","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1610.06258"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mwufi/fast-weights","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/GokuMohandas/fast-weights","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jiamings/fast-weights","reach":null}],"summary":{"ran_honours":1},"by_repo_kind":{"listed":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"1e3e05f637d7c45d","entry":"get_one_hot","repo":"jiamings/fast-weights","repo_kind":"listed","path":"generator.py","file_url":"https://github.com/jiamings/fast-weights/blob/HEAD/generator.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1e3e05f637d7c45d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}