{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/dune-dataset-for-unified-editing","title":"DUnE: Dataset for Unified Editing","arxiv_id":"2311.16087","date":"2023-11-27","proceeding":null,"authors":["Afra Feyza Akyürek","Eric Pan","Garry Kuwanto","Derry Wijaya"],"abstract":"Even the most advanced language models remain susceptible to errors necessitating to modify these models without initiating a comprehensive retraining process. Model editing refers to the modification of a model's knowledge or representations in a manner that produces the desired outcomes. Prior research primarily centered around editing factual data e.g. \"Messi plays for Inter Miami\" confining the definition of an edit to a knowledge triplet i.e. (subject, object, relation). However, as the applications of language models expand, so do the diverse ways in which we wish to edit and refine their outputs. In this study, we broaden the scope of the editing problem to include an array of editing cases such as debiasing and rectifying reasoning errors and define an edit as any natural language expression that solicits a change in the model's outputs. We are introducing DUnE-an editing benchmark where edits are natural language sentences and propose that DUnE presents a challenging yet relevant task. To substantiate this claim, we conduct an extensive series of experiments testing various editing approaches to address DUnE, demonstrating their respective strengths and weaknesses. We show that retrieval-augmented language modeling can outperform specialized editing techniques and neither set of approaches has fully solved the generalized editing problem covered by our benchmark.","url_abs":"https://arxiv.org/abs/2311.16087v1","url_pdf":"https://arxiv.org/pdf/2311.16087v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"dune-dataset-for-unified-editing","repo_url":"https://github.com/feyzaakyurek/dune","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"model-editing","task_name":"Model Editing"},{"task_slug":null,"task_name":"Triplet"}],"methods":[{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2311.16087","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2311.16087"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/feyzaakyurek/dune","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":2,"unverified":2},"by_repo_kind":{"official":{"samples":4,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"6d966b4f2613face","entry":"arc_scrape_answer","repo":"feyzaakyurek/dune","repo_kind":"official","path":"util.py","file_url":"https://github.com/feyzaakyurek/dune/blob/HEAD/util.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6d966b4f2613face"}},{"code_sha256_prefix":"34df12959f598187","entry":"new_info_scrape_answer","repo":"feyzaakyurek/dune","repo_kind":"official","path":"util.py","file_url":"https://github.com/feyzaakyurek/dune/blob/HEAD/util.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"34df12959f598187"}},{"code_sha256_prefix":"424ea2453435b59c","entry":"is_equivalent","repo":"feyzaakyurek/dune","repo_kind":"official","path":"util.py","file_url":"https://github.com/feyzaakyurek/dune/blob/HEAD/util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"424ea2453435b59c"}},{"code_sha256_prefix":"cb06e91ff992ce28","entry":"load_model_tokenizer","repo":"feyzaakyurek/dune","repo_kind":"official","path":"eval_utils.py","file_url":"https://github.com/feyzaakyurek/dune/blob/HEAD/eval_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cb06e91ff992ce28"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}