{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/memllm-finetuning-llms-to-use-an-explicit","title":"MemLLM: Finetuning LLMs to Use An Explicit Read-Write Memory","arxiv_id":"2404.11672","date":"2024-04-17","proceeding":null,"authors":["Ali Modarressi","Abdullatif Köksal","Ayyoob Imani","Mohsen Fayyaz","Hinrich Schütze"],"abstract":"While current large language models (LLMs) perform well on many knowledge-related tasks, they are limited by relying on their parameters as an implicit storage mechanism. As a result, they struggle with memorizing rare events and with updating their memory as facts change over time. In addition, the uninterpretable nature of parametric memory makes it challenging to prevent hallucination. Model editing and augmenting LLMs with parameters specialized for memory are only partial solutions. In this paper, we introduce MemLLM, a novel method of enhancing LLMs by integrating a structured and explicit read-and-write memory module. MemLLM tackles the aforementioned challenges by enabling dynamic interaction with the memory and improving the LLM's capabilities in using stored knowledge. Our experiments indicate that MemLLM enhances the LLM's performance and interpretability, in language modeling in general and knowledge-intensive tasks in particular. We see MemLLM as an important step towards making LLMs more grounded and factual through memory augmentation. The project repository is publicly available at https://github.com/amodaresi/MemLLM","url_abs":"https://arxiv.org/abs/2404.11672v3","url_pdf":"https://arxiv.org/pdf/2404.11672v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"memllm-finetuning-llms-to-use-an-explicit","repo_url":"https://github.com/amodaresi/memllm","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"hallucination","task_name":"Hallucination"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"memorization","task_name":"Memorization"},{"task_slug":"model-editing","task_name":"Model Editing"},{"task_slug":"rag","task_name":"RAG"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2404.11672","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2404.11672"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/amodaresi/memllm","reach":{"status":"ok"}}],"summary":{"ran":6,"unverified":4},"by_repo_kind":{"official":{"samples":10,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":10,"samples":[{"code_sha256_prefix":"b3a4121bc943589e","entry":"add_whitespace_around_dashes","repo":"amodaresi/memllm","repo_kind":"official","path":"inference/fast_indexing/main-model-editing.py","file_url":"https://github.com/amodaresi/memllm/blob/HEAD/inference/fast_indexing/main-model-editing.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b3a4121bc943589e"}},{"code_sha256_prefix":"e5656395e4f1b9ad","entry":"filter_","repo":"amodaresi/memllm","repo_kind":"official","path":"evaluation/model-editing/gather_zsre_result.py","file_url":"https://github.com/amodaresi/memllm/blob/HEAD/evaluation/model-editing/gather_zsre_result.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e5656395e4f1b9ad"}},{"code_sha256_prefix":"77c2cb920c5fa854","entry":"get_abstract","repo":"amodaresi/memllm","repo_kind":"official","path":"inference/fast_indexing/main-wiki.py","file_url":"https://github.com/amodaresi/memllm/blob/HEAD/inference/fast_indexing/main-wiki.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"77c2cb920c5fa854"}},{"code_sha256_prefix":"5b1744917dfd7936","entry":"get_full_text","repo":"amodaresi/memllm","repo_kind":"official","path":"inference/fast_indexing/main-wiki.py","file_url":"https://github.com/amodaresi/memllm/blob/HEAD/inference/fast_indexing/main-wiki.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5b1744917dfd7936"}},{"code_sha256_prefix":"2a48939d602b4436","entry":"get_token_span","repo":"amodaresi/memllm","repo_kind":"official","path":"evaluation/ppl-based/evaluate_v4_mistral.py","file_url":"https://github.com/amodaresi/memllm/blob/HEAD/evaluation/ppl-based/evaluate_v4_mistral.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2a48939d602b4436"}},{"code_sha256_prefix":"c38cedf562131db8","entry":"remove_between_brackets","repo":"amodaresi/memllm","repo_kind":"official","path":"evaluation/model-editing/eval_zsre.py","file_url":"https://github.com/amodaresi/memllm/blob/HEAD/evaluation/model-editing/eval_zsre.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c38cedf562131db8"}},{"code_sha256_prefix":"7cd9b4acd273b70d","entry":"convert_relation_to_readable","repo":"amodaresi/memllm","repo_kind":"official","path":"inference/fast_indexing/gatherer-redocred-testonly.py","file_url":"https://github.com/amodaresi/memllm/blob/HEAD/inference/fast_indexing/gatherer-redocred-testonly.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7cd9b4acd273b70d"}},{"code_sha256_prefix":"da93a5dc99295da7","entry":"converter","repo":"amodaresi/memllm","repo_kind":"official","path":"inference/fast_indexing/main-model-editing.py","file_url":"https://github.com/amodaresi/memllm/blob/HEAD/inference/fast_indexing/main-model-editing.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"da93a5dc99295da7"}},{"code_sha256_prefix":"c8cd0cb0e1c44f07","entry":"main","repo":"amodaresi/memllm","repo_kind":"official","path":"training/main_read_w_augmented.py","file_url":"https://github.com/amodaresi/memllm/blob/HEAD/training/main_read_w_augmented.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c8cd0cb0e1c44f07"}},{"code_sha256_prefix":"24875a8dcf1c1fdc","entry":"main","repo":"amodaresi/memllm","repo_kind":"official","path":"training/main_write_mistral_w_annotated_w_augmented.py","file_url":"https://github.com/amodaresi/memllm/blob/HEAD/training/main_write_mistral_w_annotated_w_augmented.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"24875a8dcf1c1fdc"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}