{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/lifelong-knowledge-editing-for-llms-with","title":"Lifelong Knowledge Editing for LLMs with Retrieval-Augmented Continuous Prompt Learning","arxiv_id":"2405.03279","date":"2024-05-06","proceeding":null,"authors":["Qizhou Chen","Taolin Zhang","Xiaofeng He","Dongyang Li","Chengyu Wang","Longtao Huang","Hui Xue"],"abstract":"Model editing aims to correct outdated or erroneous knowledge in large language models (LLMs) without the need for costly retraining. Lifelong model editing is the most challenging task that caters to the continuous editing requirements of LLMs. Prior works primarily focus on single or batch editing; nevertheless, these methods fall short in lifelong editing scenarios due to catastrophic knowledge forgetting and the degradation of model performance. Although retrieval-based methods alleviate these issues, they are impeded by slow and cumbersome processes of integrating the retrieved knowledge into the model. In this work, we introduce RECIPE, a RetriEval-augmented ContInuous Prompt lEarning method, to boost editing efficacy and inference efficiency in lifelong learning. RECIPE first converts knowledge statements into short and informative continuous prompts, prefixed to the LLM's input query embedding, to efficiently refine the response grounded on the knowledge. It further integrates the Knowledge Sentinel (KS) that acts as an intermediary to calculate a dynamic threshold, determining whether the retrieval repository contains relevant knowledge. Our retriever and prompt encoder are jointly trained to achieve editing properties, i.e., reliability, generality, and locality. In our experiments, RECIPE is assessed extensively across multiple LLMs and editing datasets, where it achieves superior editing performance. RECIPE also demonstrates its capability to maintain the overall performance of LLMs alongside showcasing fast editing and inference speed.","url_abs":"https://arxiv.org/abs/2405.03279v3","url_pdf":"https://arxiv.org/pdf/2405.03279v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"lifelong-knowledge-editing-for-llms-with","repo_url":"https://github.com/qizhou000/RECIPE","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"lifelong-learning","task_name":"Lifelong learning"},{"task_slug":"model-editing","task_name":"Model Editing"},{"task_slug":"prompt-learning","task_name":"Prompt Learning"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"knowledge-editing","task_name":"knowledge editing"}],"methods":[{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2405.03279","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.03279"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/qizhou000/RECIPE","reach":null}],"summary":{"ran":4,"ran_draft_wrong":3,"unverified":4},"by_repo_kind":{"official":{"samples":11,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"7af30496f8e6ecb5","entry":"KnowledgeRepModel","repo":"qizhou000/RECIPE","repo_kind":"official","path":"editors/recipe/recipe.py","file_url":"https://github.com/qizhou000/RECIPE/blob/HEAD/editors/recipe/recipe.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7af30496f8e6ecb5"}},{"code_sha256_prefix":"804969a3f054faa8","entry":"ParallelDataset","repo":"qizhou000/RECIPE","repo_kind":"official","path":"editors/recipe/recipe.py","file_url":"https://github.com/qizhou000/RECIPE/blob/HEAD/editors/recipe/recipe.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"804969a3f054faa8"}},{"code_sha256_prefix":"65642b6f8132e7c7","entry":"PromptTransformer","repo":"qizhou000/RECIPE","repo_kind":"official","path":"editors/recipe/recipe.py","file_url":"https://github.com/qizhou000/RECIPE/blob/HEAD/editors/recipe/recipe.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"65642b6f8132e7c7"}},{"code_sha256_prefix":"b84382c69b38db7f","entry":"RECIPEConfig","repo":"qizhou000/RECIPE","repo_kind":"official","path":"editors/recipe/recipe.py","file_url":"https://github.com/qizhou000/RECIPE/blob/HEAD/editors/recipe/recipe.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b84382c69b38db7f"}},{"code_sha256_prefix":"0b7bebf9d95d540c","entry":"find_module","repo":"qizhou000/RECIPE","repo_kind":"official","path":"editors/recipe/recipe.py","file_url":"https://github.com/qizhou000/RECIPE/blob/HEAD/editors/recipe/recipe.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0b7bebf9d95d540c"}},{"code_sha256_prefix":"532d15ae38761ef5","entry":"hinge_label_loss","repo":"qizhou000/RECIPE","repo_kind":"official","path":"editors/recipe/recipe.py","file_url":"https://github.com/qizhou000/RECIPE/blob/HEAD/editors/recipe/recipe.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"532d15ae38761ef5"}},{"code_sha256_prefix":"1897158165b40b18","entry":"logit_KL_loss","repo":"qizhou000/RECIPE","repo_kind":"official","path":"editors/recipe/recipe.py","file_url":"https://github.com/qizhou000/RECIPE/blob/HEAD/editors/recipe/recipe.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1897158165b40b18"}},{"code_sha256_prefix":"11dd857ade8eded8","entry":"BaseEditor","repo":"qizhou000/RECIPE","repo_kind":"official","path":"editors/recipe/recipe.py","file_url":"https://github.com/qizhou000/RECIPE/blob/HEAD/editors/recipe/recipe.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"11dd857ade8eded8"}},{"code_sha256_prefix":"972549486d7dfecb","entry":"EditorConfig","repo":"qizhou000/RECIPE","repo_kind":"official","path":"editors/recipe/recipe.py","file_url":"https://github.com/qizhou000/RECIPE/blob/HEAD/editors/recipe/recipe.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"972549486d7dfecb"}},{"code_sha256_prefix":"8c1cfd9a64ce7e2e","entry":"RECIPE","repo":"qizhou000/RECIPE","repo_kind":"official","path":"editors/recipe/recipe.py","file_url":"https://github.com/qizhou000/RECIPE/blob/HEAD/editors/recipe/recipe.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8c1cfd9a64ce7e2e"}},{"code_sha256_prefix":"062542a916d19f67","entry":"set_tokenizer_pad_id","repo":"qizhou000/RECIPE","repo_kind":"official","path":"editors/recipe/recipe.py","file_url":"https://github.com/qizhou000/RECIPE/blob/HEAD/editors/recipe/recipe.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"062542a916d19f67"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}