{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/decoupling-knowledge-from-memorization","title":"Decoupling Knowledge from Memorization: Retrieval-augmented Prompt Learning","arxiv_id":"2205.14704","date":"2022-05-29","proceeding":null,"authors":["Xiang Chen","Lei LI","Ningyu Zhang","Xiaozhuan Liang","Shumin Deng","Chuanqi Tan","Fei Huang","Luo Si","Huajun Chen"],"abstract":"Prompt learning approaches have made waves in natural language processing by inducing better few-shot performance while they still follow a parametric-based learning paradigm; the oblivion and rote memorization problems in learning may encounter unstable generalization issues. Specifically, vanilla prompt learning may struggle to utilize atypical instances by rote during fully-supervised training or overfit shallow patterns with low-shot data. To alleviate such limitations, we develop RetroPrompt with the motivation of decoupling knowledge from memorization to help the model strike a balance between generalization and memorization. In contrast with vanilla prompt learning, RetroPrompt constructs an open-book knowledge-store from training instances and implements a retrieval mechanism during the process of input, training and inference, thus equipping the model with the ability to retrieve related contexts from the training corpus as cues for enhancement. Extensive experiments demonstrate that RetroPrompt can obtain better performance in both few-shot and zero-shot settings. Besides, we further illustrate that our proposed RetroPrompt can yield better generalization abilities with new datasets. Detailed analysis of memorization indeed reveals RetroPrompt can reduce the reliance of language models on memorization; thus, improving generalization for downstream tasks. Code is available in https://github.com/zjunlp/PromptKG/tree/main/research/RetroPrompt.","url_abs":"https://arxiv.org/abs/2205.14704v5","url_pdf":"https://arxiv.org/pdf/2205.14704v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"decoupling-knowledge-from-memorization","repo_url":"https://github.com/zjunlp/promptkg","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"jax","reach":null},{"paper_slug":"decoupling-knowledge-from-memorization","repo_url":"https://github.com/zjunlp/PromptKG/tree/main/research/RetroPrompt","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"jax","reach":null}],"tasks":[{"task_slug":"few-shot-text-classification","task_name":"Few-Shot Text Classification"},{"task_slug":"memorization","task_name":"Memorization"},{"task_slug":"named-entity-recognition-ner","task_name":"Named Entity Recognition (NER)"},{"task_slug":"prompt-engineering","task_name":"Prompt Engineering"},{"task_slug":"prompt-learning","task_name":"Prompt Learning"},{"task_slug":"relation-extraction","task_name":"Relation Extraction"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"zero-shot-text-classification","task_name":"Zero-Shot Text Classification"}],"methods":[{"method_slug":"knowprompt","method_name":"KnowPrompt"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2205.14704","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2205.14704"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zjunlp/PromptKG/tree/main/research/RetroPrompt","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zjunlp/promptkg","reach":null}],"summary":{"ran_draft_wrong":1},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"c879a69ea812690e","entry":"combine_knn_and_vocab_probs","repo":"zjunlp/promptkg","repo_kind":"official","path":"research/RetroPrompt/RE_task/lit_models/transformer.py","file_url":"https://github.com/zjunlp/promptkg/blob/HEAD/research/RetroPrompt/RE_task/lit_models/transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c879a69ea812690e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}