{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/on-sample-based-explanation-methods-for-nlp","title":"On Sample Based Explanation Methods for NLP:Efficiency, Faithfulness, and Semantic Evaluation","arxiv_id":"2106.04753","date":"2021-06-09","proceeding":null,"authors":["Wei zhang","Ziming Huang","Yada Zhu","Guangnan Ye","Xiaodong Cui","Fan Zhang"],"abstract":"In the recent advances of natural language processing, the scale of the state-of-the-art models and datasets is usually extensive, which challenges the application of sample-based explanation methods in many aspects, such as explanation interpretability, efficiency, and faithfulness. In this work, for the first time, we can improve the interpretability of explanations by allowing arbitrary text sequences as the explanation unit. On top of this, we implement a hessian-free method with a model faithfulness guarantee. Finally, to compare our method with the others, we propose a semantic-based evaluation metric that can better align with humans' judgment of explanations than the widely adopted diagnostic or re-training measures. The empirical results on multiple real data sets demonstrate the proposed method's superior performance to popular explanation techniques such as Influence Function or TracIn on semantic evaluation.","url_abs":"https://arxiv.org/abs/2106.04753v1","url_pdf":"https://arxiv.org/pdf/2106.04753v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"on-sample-based-explanation-methods-for-nlp","repo_url":"https://github.com/IBM/TracInF","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"diagnostic","task_name":"Diagnostic"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2106.04753","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2106.04753"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/IBM/TracInF","reach":null}],"summary":{"ran_draft_wrong":4,"ran_fixture":1,"ran_honours":1,"unverified":4},"by_repo_kind":{"listed":{"samples":10,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"726d0822a9e90aaf","entry":"calc_loss","repo":"IBM/TracInF","repo_kind":"listed","path":"code/pif/influence_functions/hvp_grad.py","file_url":"https://github.com/IBM/TracInF/blob/HEAD/code/pif/influence_functions/hvp_grad.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"726d0822a9e90aaf"}},{"code_sha256_prefix":"f4c9f096d90e61e3","entry":"conjugate_gradient","repo":"IBM/TracInF","repo_kind":"listed","path":"code/pif/influence_functions/hvp_grad.py","file_url":"https://github.com/IBM/TracInF/blob/HEAD/code/pif/influence_functions/hvp_grad.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f4c9f096d90e61e3"}},{"code_sha256_prefix":"6ec180448a5aaa8a","entry":"grad_z","repo":"IBM/TracInF","repo_kind":"listed","path":"code/pif/influence_functions/hvp_grad.py","file_url":"https://github.com/IBM/TracInF/blob/HEAD/code/pif/influence_functions/hvp_grad.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6ec180448a5aaa8a"}},{"code_sha256_prefix":"194db402fdfd8a1d","entry":"make_functional","repo":"IBM/TracInF","repo_kind":"listed","path":"code/pif/influence_functions/hvp_grad.py","file_url":"https://github.com/IBM/TracInF/blob/HEAD/code/pif/influence_functions/hvp_grad.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"194db402fdfd8a1d"}},{"code_sha256_prefix":"5b72c46a260d3754","entry":"parameters_to_vector","repo":"IBM/TracInF","repo_kind":"listed","path":"code/pif/influence_functions/hvp_grad.py","file_url":"https://github.com/IBM/TracInF/blob/HEAD/code/pif/influence_functions/hvp_grad.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5b72c46a260d3754"}},{"code_sha256_prefix":"1521d8f9fa289d3f","entry":"tensor_to_tuple","repo":"IBM/TracInF","repo_kind":"listed","path":"code/pif/influence_functions/hvp_grad.py","file_url":"https://github.com/IBM/TracInF/blob/HEAD/code/pif/influence_functions/hvp_grad.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1521d8f9fa289d3f"}},{"code_sha256_prefix":"07957dd70a5d4a77","entry":"del_attr","repo":"IBM/TracInF","repo_kind":"listed","path":"code/pif/influence_functions/hvp_grad.py","file_url":"https://github.com/IBM/TracInF/blob/HEAD/code/pif/influence_functions/hvp_grad.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"07957dd70a5d4a77"}},{"code_sha256_prefix":"046062e4ea7ca7df","entry":"load_weights","repo":"IBM/TracInF","repo_kind":"listed","path":"code/pif/influence_functions/hvp_grad.py","file_url":"https://github.com/IBM/TracInF/blob/HEAD/code/pif/influence_functions/hvp_grad.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"046062e4ea7ca7df"}},{"code_sha256_prefix":"78cbcb281155ca8c","entry":"s_test_cg","repo":"IBM/TracInF","repo_kind":"listed","path":"code/pif/influence_functions/hvp_grad.py","file_url":"https://github.com/IBM/TracInF/blob/HEAD/code/pif/influence_functions/hvp_grad.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"78cbcb281155ca8c"}},{"code_sha256_prefix":"36495d54d434df01","entry":"set_attr","repo":"IBM/TracInF","repo_kind":"listed","path":"code/pif/influence_functions/hvp_grad.py","file_url":"https://github.com/IBM/TracInF/blob/HEAD/code/pif/influence_functions/hvp_grad.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"36495d54d434df01"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}