{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-to-faithfully-rationalize-by","title":"Learning to Faithfully Rationalize by Construction","arxiv_id":"2005.00115","date":"2020-04-30","proceeding":"ACL 2020 6","authors":["Sarthak Jain","Sarah Wiegreffe","Yuval Pinter","Byron C. Wallace"],"abstract":"In many settings it is important for one to be able to understand why a model made a particular prediction. In NLP this often entails extracting snippets of an input text `responsible for' corresponding model output; when such a snippet comprises tokens that indeed informed the model's prediction, it is a faithful explanation. In some settings, faithfulness may be critical to ensure transparency. Lei et al. (2016) proposed a model to produce faithful rationales for neural text classification by defining independent snippet extraction and prediction modules. However, the discrete selection over input tokens performed by this method complicates training, leading to high variance and requiring careful hyperparameter tuning. We propose a simpler variant of this approach that provides faithful explanations by construction. In our scheme, named FRESH, arbitrary feature importance scores (e.g., gradients from a trained model) are used to induce binary labels over token inputs, which an extractor can be trained to predict. An independent classifier module is then trained exclusively on snippets provided by the extractor; these snippets thus constitute faithful explanations, even if the classifier is arbitrarily complex. In both automatic and manual evaluations we find that variants of this simple framework yield predictive performance superior to `end-to-end' approaches, while being more general and easier to train. Code is available at https://github.com/successar/FRESH","url_abs":"https://arxiv.org/abs/2005.00115v1","url_pdf":"https://arxiv.org/pdf/2005.00115v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-to-faithfully-rationalize-by","repo_url":"https://github.com/successar/FRESH","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"learning-to-faithfully-rationalize-by","repo_url":"https://github.com/arolihas/interpretable_deepPSL","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"feature-importance","task_name":"Feature Importance"},{"task_slug":"text-classification","task_name":"Text Classification"},{"task_slug":"text-classification-1","task_name":"text-classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2005.00115","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2005.00115"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/arolihas/interpretable_deepPSL","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/successar/FRESH","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":4,"unverified":10},"by_repo_kind":{"official":{"samples":14,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"2843042d2c0e6f50","entry":"annotations_from_jsonl","repo":"successar/FRESH","repo_kind":"official","path":"Datasets/utils.py","file_url":"https://github.com/successar/FRESH/blob/HEAD/Datasets/utils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2843042d2c0e6f50"}},{"code_sha256_prefix":"b6c1438a7ae7e272","entry":"kuma_moments","repo":"successar/FRESH","repo_kind":"official","path":"Rationale_Analysis/models/kuma.py","file_url":"https://github.com/successar/FRESH/blob/HEAD/Rationale_Analysis/models/kuma.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b6c1438a7ae7e272"}},{"code_sha256_prefix":"e1878c4a3bf022ae","entry":"load_datasets","repo":"successar/FRESH","repo_kind":"official","path":"Datasets/utils.py","file_url":"https://github.com/successar/FRESH/blob/HEAD/Datasets/utils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e1878c4a3bf022ae"}},{"code_sha256_prefix":"49077456b34beefd","entry":"load_jsonl","repo":"successar/FRESH","repo_kind":"official","path":"Datasets/utils.py","file_url":"https://github.com/successar/FRESH/blob/HEAD/Datasets/utils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"49077456b34beefd"}},{"code_sha256_prefix":"70170b65c8a6c5e9","entry":"computer","repo":"successar/FRESH","repo_kind":"official","path":"human_amt/compute_amt_statistics.py","file_url":"https://github.com/successar/FRESH/blob/HEAD/human_amt/compute_amt_statistics.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"70170b65c8a6c5e9"}},{"code_sha256_prefix":"8f6eacf222a4c3f7","entry":"convert_to_tensor","repo":"successar/FRESH","repo_kind":"official","path":"Rationale_Analysis/models/kuma.py","file_url":"https://github.com/successar/FRESH/blob/HEAD/Rationale_Analysis/models/kuma.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8f6eacf222a4c3f7"}},{"code_sha256_prefix":"8ddd3bcf97fbbcd0","entry":"extract_raw_abstract","repo":"successar/FRESH","repo_kind":"official","path":"Datasets/evinf/process_evidence_inference.py","file_url":"https://github.com/successar/FRESH/blob/HEAD/Datasets/evinf/process_evidence_inference.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8ddd3bcf97fbbcd0"}},{"code_sha256_prefix":"66de11959336aeb2","entry":"generate_report","repo":"successar/FRESH","repo_kind":"official","path":"Rationale_Analysis/experiments/allentune_plot.py","file_url":"https://github.com/successar/FRESH/blob/HEAD/Rationale_Analysis/experiments/allentune_plot.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"66de11959336aeb2"}},{"code_sha256_prefix":"aee97053c9473179","entry":"kuma_mean","repo":"successar/FRESH","repo_kind":"official","path":"Rationale_Analysis/models/kuma.py","file_url":"https://github.com/successar/FRESH/blob/HEAD/Rationale_Analysis/models/kuma.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aee97053c9473179"}},{"code_sha256_prefix":"ff594b5da33b1042","entry":"main","repo":"successar/FRESH","repo_kind":"official","path":"Datasets/get_dataset_statistics.py","file_url":"https://github.com/successar/FRESH/blob/HEAD/Datasets/get_dataset_statistics.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ff594b5da33b1042"}},{"code_sha256_prefix":"9a0ae9e831dc0d57","entry":"mapper","repo":"successar/FRESH","repo_kind":"official","path":"human_amt/compute_amt_statistics.py","file_url":"https://github.com/successar/FRESH/blob/HEAD/human_amt/compute_amt_statistics.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9a0ae9e831dc0d57"}},{"code_sha256_prefix":"091c323ea322b6ab","entry":"samplemax","repo":"successar/FRESH","repo_kind":"official","path":"Rationale_Analysis/experiments/allentune_plot.py","file_url":"https://github.com/successar/FRESH/blob/HEAD/Rationale_Analysis/experiments/allentune_plot.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"091c323ea322b6ab"}},{"code_sha256_prefix":"44f8471c142ddd5c","entry":"td_format","repo":"successar/FRESH","repo_kind":"official","path":"Rationale_Analysis/experiments/allentune_plot.py","file_url":"https://github.com/successar/FRESH/blob/HEAD/Rationale_Analysis/experiments/allentune_plot.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"44f8471c142ddd5c"}},{"code_sha256_prefix":"cd1019632838faed","entry":"vote","repo":"successar/FRESH","repo_kind":"official","path":"human_amt/compute_amt_statistics.py","file_url":"https://github.com/successar/FRESH/blob/HEAD/human_amt/compute_amt_statistics.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cd1019632838faed"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}