{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/uniform-memory-retrieval-with-larger-capacity","title":"Uniform Memory Retrieval with Larger Capacity for Modern Hopfield Models","arxiv_id":"2404.03827","date":"2024-04-04","proceeding":null,"authors":["Dennis Wu","Jerry Yao-Chieh Hu","Teng-Yun Hsiao","Han Liu"],"abstract":"We propose a two-stage memory retrieval dynamics for modern Hopfield models, termed $\\mathtt{U\\text{-}Hop}$, with enhanced memory capacity. Our key contribution is a learnable feature map $\\Phi$ which transforms the Hopfield energy function into kernel space. This transformation ensures convergence between the local minima of energy and the fixed points of retrieval dynamics within the kernel space. Consequently, the kernel norm induced by $\\Phi$ serves as a novel similarity measure. It utilizes the stored memory patterns as learning data to enhance memory capacity across all modern Hopfield models. Specifically, we accomplish this by constructing a separation loss $\\mathcal{L}_\\Phi$ that separates the local minima of kernelized energy by separating stored memory patterns in kernel space. Methodologically, $\\mathtt{U\\text{-}Hop}$ memory retrieval process consists of: (Stage I) minimizing separation loss for a more uniform memory (local minimum) distribution, followed by (Stage II) standard Hopfield energy minimization for memory retrieval. This results in a significant reduction of possible metastable states in the Hopfield energy function, thus enhancing memory capacity by preventing memory confusion. Empirically, with real-world datasets, we demonstrate that $\\mathtt{U\\text{-}Hop}$ outperforms all existing modern Hopfield models and state-of-the-art similarity measures, achieving substantial improvements in both associative memory retrieval and deep learning tasks. Code is available at https://github.com/MAGICS-LAB/UHop ; future updates are on arXiv:2404.03827","url_abs":"https://arxiv.org/abs/2404.03827v3","url_pdf":"https://arxiv.org/pdf/2404.03827v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"uniform-memory-retrieval-with-larger-capacity","repo_url":"https://github.com/magics-lab/uhop","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"retrieval","task_name":"Retrieval"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2404.03827","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2404.03827"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/magics-lab/uhop","reach":null}],"summary":{"ran":4,"ran_draft_wrong":2,"unverified":1},"by_repo_kind":{"official":{"samples":7,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"7b1cf2534617049b","entry":"Association","repo":"magics-lab/uhop","repo_kind":"official","path":"layers.py","file_url":"https://github.com/magics-lab/uhop/blob/HEAD/layers.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7b1cf2534617049b"}},{"code_sha256_prefix":"5664657431b7e7a4","entry":"LearnableHopfield","repo":"magics-lab/uhop","repo_kind":"official","path":"layers.py","file_url":"https://github.com/magics-lab/uhop/blob/HEAD/layers.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5664657431b7e7a4"}},{"code_sha256_prefix":"8c413a2b668b9234","entry":"Sparsemax","repo":"magics-lab/uhop","repo_kind":"official","path":"layers.py","file_url":"https://github.com/magics-lab/uhop/blob/HEAD/layers.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8c413a2b668b9234"}},{"code_sha256_prefix":"31d60d234cc0692f","entry":"SparsemaxFunction","repo":"magics-lab/uhop","repo_kind":"official","path":"layers.py","file_url":"https://github.com/magics-lab/uhop/blob/HEAD/layers.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"31d60d234cc0692f"}},{"code_sha256_prefix":"32ac3d67805b2251","entry":"flatten_all_but_nth_dim","repo":"magics-lab/uhop","repo_kind":"official","path":"layers.py","file_url":"https://github.com/magics-lab/uhop/blob/HEAD/layers.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"32ac3d67805b2251"}},{"code_sha256_prefix":"bdec64ff3f52a1cc","entry":"unflatten_all_but_nth_dim","repo":"magics-lab/uhop","repo_kind":"official","path":"layers.py","file_url":"https://github.com/magics-lab/uhop/blob/HEAD/layers.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bdec64ff3f52a1cc"}},{"code_sha256_prefix":"827fc074dd33abc1","entry":"sqdiff","repo":"magics-lab/uhop","repo_kind":"official","path":"memory_retrieval_max_loss.py","file_url":"https://github.com/magics-lab/uhop/blob/HEAD/memory_retrieval_max_loss.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"827fc074dd33abc1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}