{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/evaluating-explainable-ai-which-algorithmic","title":"Evaluating Explainable AI: Which Algorithmic Explanations Help Users Predict Model Behavior?","arxiv_id":"2005.01831","date":"2020-05-04","proceeding":"ACL 2020 6","authors":["Peter Hase","Mohit Bansal"],"abstract":"Algorithmic approaches to interpreting machine learning models have proliferated in recent years. We carry out human subject tests that are the first of their kind to isolate the effect of algorithmic explanations on a key aspect of model interpretability, simulatability, while avoiding important confounding experimental factors. A model is simulatable when a person can predict its behavior on new inputs. Through two kinds of simulation tests involving text and tabular data, we evaluate five explanations methods: (1) LIME, (2) Anchor, (3) Decision Boundary, (4) a Prototype model, and (5) a Composite approach that combines explanations from each method. Clear evidence of method effectiveness is found in very few cases: LIME improves simulatability in tabular classification, and our Prototype method is effective in counterfactual simulation tests. We also collect subjective ratings of explanations, but we do not find that ratings are predictive of how helpful explanations are. Our results provide the first reliable and comprehensive estimates of how explanations influence simulatability across a variety of explanation methods and data domains. We show that (1) we need to be careful about the metrics we use to evaluate explanation methods, and (2) there is significant room for improvement in current methods. All our supporting code, data, and models are publicly available at: https://github.com/peterbhase/InterpretableNLP-ACL2020","url_abs":"https://arxiv.org/abs/2005.01831v1","url_pdf":"https://arxiv.org/pdf/2005.01831v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"evaluating-explainable-ai-which-algorithmic","repo_url":"https://github.com/peterbhase/InterpretableNLP-ACL2020","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":null,"task_name":"counterfactual"},{"task_slug":"tabular-classification","task_name":"tabular-classification"}],"methods":[{"method_slug":"lime","method_name":"LIME"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2005.01831","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2005.01831"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/peterbhase/InterpretableNLP-ACL2020","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":9},"by_repo_kind":{"official":{"samples":9,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"3fb4c9bcfe98d3f6","entry":"get_balanced_batches","repo":"peterbhase/InterpretableNLP-ACL2020","repo_kind":"official","path":"tabular/gather-experiment-data.py","file_url":"https://github.com/peterbhase/InterpretableNLP-ACL2020/blob/HEAD/tabular/gather-experiment-data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3fb4c9bcfe98d3f6"}},{"code_sha256_prefix":"ae7b52b915114f8d","entry":"get_balanced_batches","repo":"peterbhase/InterpretableNLP-ACL2020","repo_kind":"official","path":"text/gather-experiment-data.py","file_url":"https://github.com/peterbhase/InterpretableNLP-ACL2020/blob/HEAD/text/gather-experiment-data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ae7b52b915114f8d"}},{"code_sha256_prefix":"e8dfabee12cd3fcf","entry":"load_pickles","repo":"peterbhase/InterpretableNLP-ACL2020","repo_kind":"official","path":"tabular/anchor/make_graphs_and_table.py","file_url":"https://github.com/peterbhase/InterpretableNLP-ACL2020/blob/HEAD/tabular/anchor/make_graphs_and_table.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e8dfabee12cd3fcf"}},{"code_sha256_prefix":"60da6efa17ee6dc2","entry":"make_graph","repo":"peterbhase/InterpretableNLP-ACL2020","repo_kind":"official","path":"tabular/anchor/make_graphs_and_table.py","file_url":"https://github.com/peterbhase/InterpretableNLP-ACL2020/blob/HEAD/tabular/anchor/make_graphs_and_table.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"60da6efa17ee6dc2"}},{"code_sha256_prefix":"1db3f658ae6ec3e0","entry":"make_table","repo":"peterbhase/InterpretableNLP-ACL2020","repo_kind":"official","path":"tabular/anchor/make_graphs_and_table.py","file_url":"https://github.com/peterbhase/InterpretableNLP-ACL2020/blob/HEAD/tabular/anchor/make_graphs_and_table.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1db3f658ae6ec3e0"}},{"code_sha256_prefix":"e1b1e0837af55402","entry":"matrix_subset","repo":"peterbhase/InterpretableNLP-ACL2020","repo_kind":"official","path":"tabular/anchor/anchor_base.py","file_url":"https://github.com/peterbhase/InterpretableNLP-ACL2020/blob/HEAD/tabular/anchor/anchor_base.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e1b1e0837af55402"}},{"code_sha256_prefix":"583b76ee9dc07214","entry":"pick_perturbation","repo":"peterbhase/InterpretableNLP-ACL2020","repo_kind":"official","path":"tabular/gather-experiment-data.py","file_url":"https://github.com/peterbhase/InterpretableNLP-ACL2020/blob/HEAD/tabular/gather-experiment-data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"583b76ee9dc07214"}},{"code_sha256_prefix":"65318aa5dc0ea291","entry":"pick_perturbation","repo":"peterbhase/InterpretableNLP-ACL2020","repo_kind":"official","path":"text/gather-experiment-data.py","file_url":"https://github.com/peterbhase/InterpretableNLP-ACL2020/blob/HEAD/text/gather-experiment-data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"65318aa5dc0ea291"}},{"code_sha256_prefix":"3d17a26510e578cb","entry":"random_until_k","repo":"peterbhase/InterpretableNLP-ACL2020","repo_kind":"official","path":"tabular/anchor/process_results.py","file_url":"https://github.com/peterbhase/InterpretableNLP-ACL2020/blob/HEAD/tabular/anchor/process_results.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3d17a26510e578cb"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}