{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/exploring-the-efficacy-of-automatically","title":"Exploring the Efficacy of Automatically Generated Counterfactuals for Sentiment Analysis","arxiv_id":"2106.15231","date":"2021-06-29","proceeding":"ACL 2021 5","authors":["Linyi Yang","Jiazheng Li","Pádraig Cunningham","Yue Zhang","Barry Smyth","Ruihai Dong"],"abstract":"While state-of-the-art NLP models have been achieving the excellent performance of a wide range of tasks in recent years, important questions are being raised about their robustness and their underlying sensitivity to systematic biases that may exist in their training and test data. Such issues come to be manifest in performance problems when faced with out-of-distribution data in the field. One recent solution has been to use counterfactually augmented datasets in order to reduce any reliance on spurious patterns that may exist in the original data. Producing high-quality augmented data can be costly and time-consuming as it usually needs to involve human feedback and crowdsourcing efforts. In this work, we propose an alternative by describing and evaluating an approach to automatically generating counterfactual data for data augmentation and explanation. A comprehensive evaluation on several different datasets and using a variety of state-of-the-art benchmarks demonstrate how our approach can achieve significant improvements in model performance when compared to models training on the original data and even when compared to models trained with the benefit of human-generated augmented data.","url_abs":"https://arxiv.org/abs/2106.15231v3","url_pdf":"https://arxiv.org/pdf/2106.15231v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"exploring-the-efficacy-of-automatically","repo_url":"https://github.com/lijiazheng99/Counterfactuals-for-Sentiment-Analysis","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"data-augmentation","task_name":"Data Augmentation"},{"task_slug":"sentiment-analysis","task_name":"Sentiment Analysis"},{"task_slug":null,"task_name":"counterfactual"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2106.15231","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2106.15231"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lijiazheng99/Counterfactuals-for-Sentiment-Analysis","reach":null}],"summary":{"ran":2},"by_repo_kind":{"official":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"10bbdfd4b56383ef","entry":"Cfsa","repo":"lijiazheng99/Counterfactuals-for-Sentiment-Analysis","repo_kind":"official","path":"cfsa/cfsarep.py","file_url":"https://github.com/lijiazheng99/Counterfactuals-for-Sentiment-Analysis/blob/HEAD/cfsa/cfsarep.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"10bbdfd4b56383ef"}},{"code_sha256_prefix":"03fc05d283141c92","entry":"Cfsarep","repo":"lijiazheng99/Counterfactuals-for-Sentiment-Analysis","repo_kind":"official","path":"cfsa/cfsarep.py","file_url":"https://github.com/lijiazheng99/Counterfactuals-for-Sentiment-Analysis/blob/HEAD/cfsa/cfsarep.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"03fc05d283141c92"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}