{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/search-methods-for-sufficient-socially","title":"The Out-of-Distribution Problem in Explainability and Search Methods for Feature Importance Explanations","arxiv_id":"2106.00786","date":"2021-06-01","proceeding":"NeurIPS 2021 12","authors":["Peter Hase","Harry Xie","Mohit Bansal"],"abstract":"Feature importance (FI) estimates are a popular form of explanation, and they are commonly created and evaluated by computing the change in model confidence caused by removing certain input features at test time. For example, in the standard Sufficiency metric, only the top-k most important tokens are kept. In this paper, we study several under-explored dimensions of FI explanations, providing conceptual and empirical improvements for this form of explanation. First, we advance a new argument for why it can be problematic to remove features from an input when creating or evaluating explanations: the fact that these counterfactual inputs are out-of-distribution (OOD) to models implies that the resulting explanations are socially misaligned. The crux of the problem is that the model prior and random weight initialization influence the explanations (and explanation metrics) in unintended ways. To resolve this issue, we propose a simple alteration to the model training process, which results in more socially aligned explanations and metrics. Second, we compare among five approaches for removing features from model inputs. We find that some methods produce more OOD counterfactuals than others, and we make recommendations for selecting a feature-replacement function. Finally, we introduce four search-based methods for identifying FI explanations and compare them to strong baselines, including LIME, Anchors, and Integrated Gradients. Through experiments with six diverse text classification datasets, we find that the only method that consistently outperforms random search is a Parallel Local Search (PLS) that we introduce. Improvements over the second-best method are as large as 5.4 points for Sufficiency and 17 points for Comprehensiveness. All supporting code for experiments in this paper is publicly available at https://github.com/peterbhase/ExplanationSearch.","url_abs":"https://arxiv.org/abs/2106.00786v2","url_pdf":"https://arxiv.org/pdf/2106.00786v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"search-methods-for-sufficient-socially","repo_url":"https://github.com/peterbhase/ExplanationSearch","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"feature-importance","task_name":"Feature Importance"},{"task_slug":"text-classification","task_name":"Text Classification"},{"task_slug":null,"task_name":"counterfactual"},{"task_slug":"text-classification-1","task_name":"text-classification"}],"methods":[{"method_slug":"counterfactuals","method_name":"Counterfactuals"},{"method_slug":"lime","method_name":"LIME"},{"method_slug":"random-search","method_name":"Random Search"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2106.00786","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2106.00786"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/peterbhase/ExplanationSearch","reach":null}],"summary":{"ran":1,"unverified":3},"by_repo_kind":{"official":{"samples":4,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"297c47761a4ab839","entry":"PLSExplainer","repo":"peterbhase/ExplanationSearch","repo_kind":"official","path":"explanation_methods_and_eval/src/models/parallel_local_search.py","file_url":"https://github.com/peterbhase/ExplanationSearch/blob/HEAD/explanation_methods_and_eval/src/models/parallel_local_search.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"297c47761a4ab839"}},{"code_sha256_prefix":"b5f573fe9b7a4f5e","entry":"compute_counterfactual","repo":"peterbhase/explanationsearch","repo_kind":"official","path":"OOD_analysis/compute_counterfactual.py","file_url":"https://github.com/peterbhase/explanationsearch/blob/HEAD/OOD_analysis/compute_counterfactual.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b5f573fe9b7a4f5e"}},{"code_sha256_prefix":"f26e2899dfb5e361","entry":"get_masked_inputs","repo":"peterbhase/explanationsearch","repo_kind":"official","path":"OOD_analysis/compute_counterfactual.py","file_url":"https://github.com/peterbhase/explanationsearch/blob/HEAD/OOD_analysis/compute_counterfactual.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f26e2899dfb5e361"}},{"code_sha256_prefix":"4f898c3ab836a9b9","entry":"random_mask","repo":"peterbhase/explanationsearch","repo_kind":"official","path":"OOD_analysis/compute_counterfactual.py","file_url":"https://github.com/peterbhase/explanationsearch/blob/HEAD/OOD_analysis/compute_counterfactual.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4f898c3ab836a9b9"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}