{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/are-all-spurious-features-in-natural-language","title":"Are All Spurious Features in Natural Language Alike? An Analysis through a Causal Lens","arxiv_id":"2210.14011","date":"2022-10-25","proceeding":null,"authors":["Nitish Joshi","Xiang Pan","He He"],"abstract":"The term `spurious correlations' has been used in NLP to informally denote any undesirable feature-label correlations. However, a correlation can be undesirable because (i) the feature is irrelevant to the label (e.g. punctuation in a review), or (ii) the feature's effect on the label depends on the context (e.g. negation words in a review), which is ubiquitous in language tasks. In case (i), we want the model to be invariant to the feature, which is neither necessary nor sufficient for prediction. But in case (ii), even an ideal model (e.g. humans) must rely on the feature, since it is necessary (but not sufficient) for prediction. Therefore, a more fine-grained treatment of spurious features is needed to specify the desired model behavior. We formalize this distinction using a causal model and probabilities of necessity and sufficiency, which delineates the causal relations between a feature and a label. We then show that this distinction helps explain results of existing debiasing methods on different spurious features, and demystifies surprising results such as the encoding of spurious features in model representations after debiasing.","url_abs":"https://arxiv.org/abs/2210.14011v1","url_pdf":"https://arxiv.org/pdf/2210.14011v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"are-all-spurious-features-in-natural-language","repo_url":"https://github.com/joshinh/spurious-correlations-nlp","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"all","task_name":"All"},{"task_slug":"negation","task_name":"Negation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2210.14011","atlas_url":"https://app.syntology.ai/?focus=2210.14011","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2210.14011"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/joshinh/spurious-correlations-nlp","reach":null}],"summary":{"ran_draft_wrong":4,"ran_honours":1,"unverified":2},"by_repo_kind":{"official":{"samples":7,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"83b8335aa82f90c4","entry":"build_probe","repo":"joshinh/spurious-correlations-nlp","repo_kind":"official","path":"nlp/scripts/probing.py","file_url":"https://github.com/joshinh/spurious-correlations-nlp/blob/HEAD/nlp/scripts/probing.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"83b8335aa82f90c4"}},{"code_sha256_prefix":"323fa71e27b117eb","entry":"compute_metrics_default","repo":"joshinh/spurious-correlations-nlp","repo_kind":"official","path":"nlp/scripts/training.py","file_url":"https://github.com/joshinh/spurious-correlations-nlp/blob/HEAD/nlp/scripts/training.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"323fa71e27b117eb"}},{"code_sha256_prefix":"6603d34965799b5d","entry":"get_projection_to_intersection_of_nullspaces","repo":"joshinh/spurious-correlations-nlp","repo_kind":"official","path":"nlp/scripts/debias.py","file_url":"https://github.com/joshinh/spurious-correlations-nlp/blob/HEAD/nlp/scripts/debias.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6603d34965799b5d"}},{"code_sha256_prefix":"109a64c49d7364c5","entry":"get_rowspace_projection","repo":"joshinh/spurious-correlations-nlp","repo_kind":"official","path":"nlp/scripts/debias.py","file_url":"https://github.com/joshinh/spurious-correlations-nlp/blob/HEAD/nlp/scripts/debias.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"109a64c49d7364c5"}},{"code_sha256_prefix":"02436469ae12f271","entry":"per_class_accuracy_with_names","repo":"joshinh/spurious-correlations-nlp","repo_kind":"official","path":"nlp/scripts/training.py","file_url":"https://github.com/joshinh/spurious-correlations-nlp/blob/HEAD/nlp/scripts/training.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"02436469ae12f271"}},{"code_sha256_prefix":"3b1da49bcea7e00a","entry":"debias_by_specific_directions","repo":"joshinh/spurious-correlations-nlp","repo_kind":"official","path":"nlp/scripts/debias.py","file_url":"https://github.com/joshinh/spurious-correlations-nlp/blob/HEAD/nlp/scripts/debias.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3b1da49bcea7e00a"}},{"code_sha256_prefix":"b2a4cae875a44fdc","entry":"paired_accuracy","repo":"joshinh/spurious-correlations-nlp","repo_kind":"official","path":"nlp/scripts/training.py","file_url":"https://github.com/joshinh/spurious-correlations-nlp/blob/HEAD/nlp/scripts/training.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b2a4cae875a44fdc"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}