{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/know-how-to-make-up-your-mind-adversarially","title":"KNOW How to Make Up Your Mind! Adversarially Detecting and Alleviating Inconsistencies in Natural Language Explanations","arxiv_id":"2306.02980","date":"2023-06-05","proceeding":null,"authors":["Myeongjun Jang","Bodhisattwa Prasad Majumder","Julian McAuley","Thomas Lukasiewicz","Oana-Maria Camburu"],"abstract":"While recent works have been considerably improving the quality of the natural language explanations (NLEs) generated by a model to justify its predictions, there is very limited research in detecting and alleviating inconsistencies among generated NLEs. In this work, we leverage external knowledge bases to significantly improve on an existing adversarial attack for detecting inconsistent NLEs. We apply our attack to high-performing NLE models and show that models with higher NLE quality do not necessarily generate fewer inconsistencies. Moreover, we propose an off-the-shelf mitigation method to alleviate inconsistencies by grounding the model into external background knowledge. Our method decreases the inconsistencies of previous high-performing NLE models as detected by our attack.","url_abs":"https://arxiv.org/abs/2306.02980v1","url_pdf":"https://arxiv.org/pdf/2306.02980v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"know-how-to-make-up-your-mind-adversarially","repo_url":"https://github.com/mj-jang/eknowia","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"adversarial-attack","task_name":"Adversarial Attack"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2306.02980","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2306.02980"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/MJ-Jang/eKnowIA","reach":null}],"summary":{"ran_draft_wrong":1,"unverified":3},"by_repo_kind":{"official":{"samples":4,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"ada051795ff85e07","entry":"extract_from_cn","repo":"MJ-Jang/eKnowIA","repo_kind":"official","path":"src/generate_inconsistent_expl.py","file_url":"https://github.com/MJ-Jang/eKnowIA/blob/HEAD/src/generate_inconsistent_expl.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ada051795ff85e07"}},{"code_sha256_prefix":"e0d2650cc1179b4b","entry":"antonyms_wn","repo":"MJ-Jang/eKnowIA","repo_kind":"official","path":"src/generate_inconsistent_expl.py","file_url":"https://github.com/MJ-Jang/eKnowIA/blob/HEAD/src/generate_inconsistent_expl.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e0d2650cc1179b4b"}},{"code_sha256_prefix":"055608e648d56ca9","entry":"get_synonyms_from_google","repo":"MJ-Jang/eKnowIA","repo_kind":"official","path":"src/generate_inconsistent_expl.py","file_url":"https://github.com/MJ-Jang/eKnowIA/blob/HEAD/src/generate_inconsistent_expl.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"055608e648d56ca9"}},{"code_sha256_prefix":"38c3aaa21bacd553","entry":"replace_antonym","repo":"MJ-Jang/eKnowIA","repo_kind":"official","path":"src/generate_inconsistent_expl.py","file_url":"https://github.com/MJ-Jang/eKnowIA/blob/HEAD/src/generate_inconsistent_expl.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"38c3aaa21bacd553"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}