{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/untying-the-reversal-curse-via-bidirectional","title":"Untying the Reversal Curse via Bidirectional Language Model Editing","arxiv_id":"2310.10322","date":"2023-10-16","proceeding":null,"authors":["Jun-Yu Ma","Jia-Chen Gu","Zhen-Hua Ling","Quan Liu","Cong Liu"],"abstract":"Recent studies have demonstrated that large language models (LLMs) store massive factual knowledge within their parameters. But existing LLMs are prone to hallucinate unintended text due to false or outdated knowledge. Since retraining LLMs is resource intensive, there has been a growing interest in the concept of model editing. Despite the emergence of benchmarks and approaches, these unidirectional editing and evaluation have failed to explore the reversal curse. Intuitively, if \"The capital of France is\" is edited to be a counterfact \"London\" within a model, then it should be able to naturally reason and recall the reverse fact, i.e., \"London is the capital of\" followed by \"France\" instead of \"England\". In this paper, we study bidirectional language model editing, aiming to provide rigorous model editing evaluation to assess if edited LLMs can recall the editing knowledge bidirectionally. A new evaluation metric of reversibility is introduced, and a benchmark dubbed as Bidirectional Assessment for Knowledge Editing (BAKE) is constructed to evaluate the reversibility of edited models in recalling knowledge in the reverse direction of editing. We surprisingly observe that while current editing methods and LLMs can effectively recall editing facts in the direction of editing, they suffer serious deficiencies when evaluated in the reverse direction. To mitigate the reversal curse, a method named Bidirectionally Inversible Relationship moDeling (BIRD) is proposed. A set of editing objectives that incorporate bidirectional relationships between subject and object into the updated model weights are designed. Experiments show that BIRD improves the performance of four representative LLMs of different sizes via question answering and judgement.","url_abs":"https://arxiv.org/abs/2310.10322v2","url_pdf":"https://arxiv.org/pdf/2310.10322v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"untying-the-reversal-curse-via-bidirectional","repo_url":"https://github.com/mjy1111/bake","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"model-editing","task_name":"Model Editing"},{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"knowledge-editing","task_name":"knowledge editing"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2310.10322","atlas_url":"https://app.syntology.ai/?focus=2310.10322","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.10322"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mjy1111/bake","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":1,"ran_fixture":1,"unverified":3},"by_repo_kind":{"official":{"samples":5,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"27659a9c234ffb11","entry":"binary_log_probs","repo":"mjy1111/bake","repo_kind":"official","path":"easyeditor/trainer/losses.py","file_url":"https://github.com/mjy1111/bake/blob/HEAD/easyeditor/trainer/losses.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"27659a9c234ffb11"}},{"code_sha256_prefix":"e078eeed20f03838","entry":"masked_mean","repo":"mjy1111/bake","repo_kind":"official","path":"easyeditor/trainer/losses.py","file_url":"https://github.com/mjy1111/bake/blob/HEAD/easyeditor/trainer/losses.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e078eeed20f03838"}},{"code_sha256_prefix":"d2e4c1979dd70b9a","entry":"get_model","repo":"mjy1111/bake","repo_kind":"official","path":"easyeditor/trainer/models.py","file_url":"https://github.com/mjy1111/bake/blob/HEAD/easyeditor/trainer/models.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d2e4c1979dd70b9a"}},{"code_sha256_prefix":"2defe42cf1a7f301","entry":"get_tokenizer","repo":"mjy1111/bake","repo_kind":"official","path":"easyeditor/trainer/models.py","file_url":"https://github.com/mjy1111/bake/blob/HEAD/easyeditor/trainer/models.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2defe42cf1a7f301"}},{"code_sha256_prefix":"bcc592aa5c1e2bb3","entry":"kl_loc_loss","repo":"mjy1111/bake","repo_kind":"official","path":"easyeditor/trainer/losses.py","file_url":"https://github.com/mjy1111/bake/blob/HEAD/easyeditor/trainer/losses.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bcc592aa5c1e2bb3"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}