{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/universal-hopfield-networks-a-general","title":"Universal Hopfield Networks: A General Framework for Single-Shot Associative Memory Models","arxiv_id":"2202.04557","date":"2022-02-09","proceeding":null,"authors":["Beren Millidge","Tommaso Salvatori","Yuhang Song","Thomas Lukasiewicz","Rafal Bogacz"],"abstract":"A large number of neural network models of associative memory have been proposed in the literature. These include the classical Hopfield networks (HNs), sparse distributed memories (SDMs), and more recently the modern continuous Hopfield networks (MCHNs), which possesses close links with self-attention in machine learning. In this paper, we propose a general framework for understanding the operation of such memory networks as a sequence of three operations: similarity, separation, and projection. We derive all these memory models as instances of our general framework with differing similarity and separation functions. We extend the mathematical framework of Krotov et al (2020) to express general associative memory models using neural network dynamics with only second-order interactions between neurons, and derive a general energy function that is a Lyapunov function of the dynamics. Finally, using our framework, we empirically investigate the capacity of using different similarity functions for these associative memory models, beyond the dot product similarity measure, and demonstrate empirically that Euclidean or Manhattan distance similarity metrics perform substantially better in practice on many tasks, enabling a more robust retrieval and higher memory capacity than existing models.","url_abs":"https://arxiv.org/abs/2202.04557v2","url_pdf":"https://arxiv.org/pdf/2202.04557v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"universal-hopfield-networks-a-general","repo_url":"https://github.com/BerenMillidge/Theory_Associative_Memory","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"retrieval","task_name":"Retrieval"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2202.04557","atlas_url":"https://app.syntology.ai/?focus=2202.04557","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2202.04557"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/BerenMillidge/Theory_Associative_Memory","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":5,"unverified":2},"by_repo_kind":{"official":{"samples":7,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"7dea47c997618e3f","entry":"gaussian_perturb_image","repo":"BerenMillidge/Theory_Associative_Memory","repo_kind":"official","path":"functions.py","file_url":"https://github.com/BerenMillidge/Theory_Associative_Memory/blob/HEAD/functions.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7dea47c997618e3f"}},{"code_sha256_prefix":"f977b3edab4a82bf","entry":"get_id_dictionary","repo":"BerenMillidge/Theory_Associative_Memory","repo_kind":"official","path":"data.py","file_url":"https://github.com/BerenMillidge/Theory_Associative_Memory/blob/HEAD/data.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f977b3edab4a82bf"}},{"code_sha256_prefix":"1545064ef5ba1b5c","entry":"halve_continuous_img","repo":"BerenMillidge/Theory_Associative_Memory","repo_kind":"official","path":"functions.py","file_url":"https://github.com/BerenMillidge/Theory_Associative_Memory/blob/HEAD/functions.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1545064ef5ba1b5c"}},{"code_sha256_prefix":"748ea182254b74cd","entry":"mask_continuous_img","repo":"BerenMillidge/Theory_Associative_Memory","repo_kind":"official","path":"functions.py","file_url":"https://github.com/BerenMillidge/Theory_Associative_Memory/blob/HEAD/functions.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"748ea182254b74cd"}},{"code_sha256_prefix":"67893413bb7ad279","entry":"parse_sname_for_title","repo":"BerenMillidge/Theory_Associative_Memory","repo_kind":"official","path":"plotting.py","file_url":"https://github.com/BerenMillidge/Theory_Associative_Memory/blob/HEAD/plotting.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"67893413bb7ad279"}},{"code_sha256_prefix":"53ce2056552aef4f","entry":"get_cifar10","repo":"BerenMillidge/Theory_Associative_Memory","repo_kind":"official","path":"data.py","file_url":"https://github.com/BerenMillidge/Theory_Associative_Memory/blob/HEAD/data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"53ce2056552aef4f"}},{"code_sha256_prefix":"166095face05d8e0","entry":"load_mnist","repo":"BerenMillidge/Theory_Associative_Memory","repo_kind":"official","path":"data.py","file_url":"https://github.com/BerenMillidge/Theory_Associative_Memory/blob/HEAD/data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"166095face05d8e0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}