{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/unke-unstructured-knowledge-editing-in-large","title":"Everything is Editable: Extend Knowledge Editing to Unstructured Data in Large Language Models","arxiv_id":"2405.15349","date":"2024-05-24","proceeding":null,"authors":["Jingcheng Deng","Zihao Wei","Liang Pang","Hanxing Ding","HuaWei Shen","Xueqi Cheng"],"abstract":"Recent knowledge editing methods have primarily focused on modifying structured knowledge in large language models. However, this task setting overlooks the fact that a significant portion of real-world knowledge is stored in an unstructured format, characterized by long-form content, noise, and a complex yet comprehensive nature. Techniques like local layer key-value storage and term-driven optimization, as used in previous methods like MEMIT, are not effective for handling unstructured knowledge. To address these challenges, we propose a novel Unstructured Knowledge Editing method, namely UnKE, which extends previous assumptions in the layer dimension and token dimension. Firstly, in the layer dimension, we propose non-local block key-value storage to replace local layer key-value storage, increasing the representation ability of key-value pairs and incorporating attention layer knowledge. Secondly, in the token dimension, we replace term-driven optimization with cause-driven optimization, which edits the last token directly while preserving context, avoiding the need to locate terms and preventing the loss of context information. Results on newly proposed unstructured knowledge editing dataset (UnKEBench) and traditional structured datasets demonstrate that UnKE achieves remarkable performance, surpassing strong baselines. In addition, UnKE has robust batch editing and sequential editing capabilities.","url_abs":"https://arxiv.org/abs/2405.15349v2","url_pdf":"https://arxiv.org/pdf/2405.15349v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"unke-unstructured-knowledge-editing-in-large","repo_url":"https://github.com/TrustedLLM/UnKE","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"world-knowledge","task_name":"World Knowledge"},{"task_slug":"knowledge-editing","task_name":"knowledge editing"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"non-local-block","method_name":"Non-Local Block"},{"method_slug":"non-local-operation","method_name":"Non-Local Operation"},{"method_slug":"residual-connection","method_name":"Residual Connection"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2405.15349","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.15349"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/TrustedLLM/UnKE","reach":null}],"summary":{"ran_draft_wrong":4,"ran_honours":1,"unverified":7},"by_repo_kind":{"official":{"samples":12,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":12,"samples":[{"code_sha256_prefix":"4c479fe8591c3b1b","entry":"get_causal_mask","repo":"TrustedLLM/UnKE","repo_kind":"official","path":"code/unke.py","file_url":"https://github.com/TrustedLLM/UnKE/blob/HEAD/code/unke.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4c479fe8591c3b1b"}},{"code_sha256_prefix":"38bfd17b85d0eb13","entry":"get_optimizer_params","repo":"TrustedLLM/UnKE","repo_kind":"official","path":"code/unke.py","file_url":"https://github.com/TrustedLLM/UnKE/blob/HEAD/code/unke.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"38bfd17b85d0eb13"}},{"code_sha256_prefix":"9e86478d518093f8","entry":"get_parameter","repo":"TrustedLLM/UnKE","repo_kind":"official","path":"code/unke.py","file_url":"https://github.com/TrustedLLM/UnKE/blob/HEAD/code/unke.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9e86478d518093f8"}},{"code_sha256_prefix":"6c6b2b6309e0acf6","entry":"get_qwen2_causal_mask","repo":"TrustedLLM/UnKE","repo_kind":"official","path":"code/unke.py","file_url":"https://github.com/TrustedLLM/UnKE/blob/HEAD/code/unke.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6c6b2b6309e0acf6"}},{"code_sha256_prefix":"70f6ab8bde55420e","entry":"recursive_copy","repo":"TrustedLLM/UnKE","repo_kind":"official","path":"code/unke.py","file_url":"https://github.com/TrustedLLM/UnKE/blob/HEAD/code/unke.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"70f6ab8bde55420e"}},{"code_sha256_prefix":"b914e7641a23c5d6","entry":"Config","repo":"TrustedLLM/UnKE","repo_kind":"official","path":"code/unke.py","file_url":"https://github.com/TrustedLLM/UnKE/blob/HEAD/code/unke.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b914e7641a23c5d6"}},{"code_sha256_prefix":"c37190b098c7eb47","entry":"Trace","repo":"TrustedLLM/UnKE","repo_kind":"official","path":"code/unke.py","file_url":"https://github.com/TrustedLLM/UnKE/blob/HEAD/code/unke.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c37190b098c7eb47"}},{"code_sha256_prefix":"58964585777d5e0a","entry":"TraceDict","repo":"TrustedLLM/UnKE","repo_kind":"official","path":"code/unke.py","file_url":"https://github.com/TrustedLLM/UnKE/blob/HEAD/code/unke.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"58964585777d5e0a"}},{"code_sha256_prefix":"c225bf0a6d643a8d","entry":"compute_ks","repo":"TrustedLLM/UnKE","repo_kind":"official","path":"code/unke.py","file_url":"https://github.com/TrustedLLM/UnKE/blob/HEAD/code/unke.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c225bf0a6d643a8d"}},{"code_sha256_prefix":"d9998262dbf04f50","entry":"compute_z","repo":"TrustedLLM/UnKE","repo_kind":"official","path":"code/unke.py","file_url":"https://github.com/TrustedLLM/UnKE/blob/HEAD/code/unke.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d9998262dbf04f50"}},{"code_sha256_prefix":"0a43e20a26bbf657","entry":"execute_batch_unke","repo":"TrustedLLM/UnKE","repo_kind":"official","path":"code/unke.py","file_url":"https://github.com/TrustedLLM/UnKE/blob/HEAD/code/unke.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0a43e20a26bbf657"}},{"code_sha256_prefix":"6b1602732a4a731f","entry":"set_requires_grad","repo":"TrustedLLM/UnKE","repo_kind":"official","path":"code/unke.py","file_url":"https://github.com/TrustedLLM/UnKE/blob/HEAD/code/unke.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6b1602732a4a731f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}