{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/nile-natural-language-inference-with-faithful","title":"NILE : Natural Language Inference with Faithful Natural Language Explanations","arxiv_id":"2005.12116","date":"2020-05-25","proceeding":"ACL 2020 6","authors":["Sawan Kumar","Partha Talukdar"],"abstract":"The recent growth in the popularity and success of deep learning models on NLP classification tasks has accompanied the need for generating some form of natural language explanation of the predicted labels. Such generated natural language (NL) explanations are expected to be faithful, i.e., they should correlate well with the model's internal decision making. In this work, we focus on the task of natural language inference (NLI) and address the following question: can we build NLI systems which produce labels with high accuracy, while also generating faithful explanations of its decisions? We propose Natural-language Inference over Label-specific Explanations (NILE), a novel NLI method which utilizes auto-generated label-specific NL explanations to produce labels along with its faithful explanation. We demonstrate NILE's effectiveness over previously reported methods through automated and human evaluation of the produced labels and explanations. Our evaluation of NILE also supports the claim that accurate systems capable of providing testable explanations of their decisions can be designed. We discuss the faithfulness of NILE's explanations in terms of sensitivity of the decisions to the corresponding explanations. We argue that explicit evaluation of faithfulness, in addition to label and explanation accuracy, is an important step in evaluating model's explanations. Further, we demonstrate that task-specific probes are necessary to establish such sensitivity.","url_abs":"https://arxiv.org/abs/2005.12116v1","url_pdf":"https://arxiv.org/pdf/2005.12116v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"nile-natural-language-inference-with-faithful","repo_url":"https://github.com/SawanKumar28/nile","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"decision-making","task_name":"Decision Making"},{"task_slug":"natural-language-inference","task_name":"Natural Language Inference"},{"task_slug":"sensitivity","task_name":"Sensitivity"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2005.12116","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2005.12116"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/SawanKumar28/nile","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":5},"by_repo_kind":{"official":{"samples":5,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"8b3b42efc2937a91","entry":"convert_examples_to_features","repo":"SawanKumar28/nile","repo_kind":"official","path":"example_to_feature.py","file_url":"https://github.com/SawanKumar28/nile/blob/HEAD/example_to_feature.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8b3b42efc2937a91"}},{"code_sha256_prefix":"5c66a15b959226eb","entry":"exp_compute_metrics","repo":"SawanKumar28/nile","repo_kind":"official","path":"nli_utils.py","file_url":"https://github.com/SawanKumar28/nile/blob/HEAD/nli_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5c66a15b959226eb"}},{"code_sha256_prefix":"aa12fcd256d6dae0","entry":"get_logits","repo":"SawanKumar28/nile","repo_kind":"official","path":"run_nli.py","file_url":"https://github.com/SawanKumar28/nile/blob/HEAD/run_nli.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"aa12fcd256d6dae0"}},{"code_sha256_prefix":"fc9eea0a993da562","entry":"sample_sequence","repo":"SawanKumar28/nile","repo_kind":"official","path":"finetune_lm.py","file_url":"https://github.com/SawanKumar28/nile/blob/HEAD/finetune_lm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fc9eea0a993da562"}},{"code_sha256_prefix":"3c241ecfe3749a6d","entry":"simple_accuracy","repo":"SawanKumar28/nile","repo_kind":"official","path":"nli_utils.py","file_url":"https://github.com/SawanKumar28/nile/blob/HEAD/nli_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3c241ecfe3749a6d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}