{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-important-features-through","title":"Learning Important Features Through Propagating Activation Differences","arxiv_id":"1704.02685","date":"2017-04-10","proceeding":"ICML 2017 8","authors":["Avanti Shrikumar","Peyton Greenside","Anshul Kundaje"],"abstract":"The purported \"black box\" nature of neural networks is a barrier to adoption in applications where interpretability is essential. Here we present DeepLIFT (Deep Learning Important FeaTures), a method for decomposing the output prediction of a neural network on a specific input by backpropagating the contributions of all neurons in the network to every feature of the input. DeepLIFT compares the activation of each neuron to its 'reference activation' and assigns contribution scores according to the difference. By optionally giving separate consideration to positive and negative contributions, DeepLIFT can also reveal dependencies which are missed by other approaches. Scores can be computed efficiently in a single backward pass. We apply DeepLIFT to models trained on MNIST and simulated genomic data, and show significant advantages over gradient-based methods. Video tutorial: http://goo.gl/qKb7pL, ICML slides: bit.ly/deeplifticmlslides, ICML talk: https://vimeo.com/238275076, code: http://goo.gl/RM8jvH.","url_abs":"https://arxiv.org/abs/1704.02685v2","url_pdf":"https://arxiv.org/pdf/1704.02685v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-important-features-through","repo_url":"https://github.com/adrhill/explainableai.jl","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"learning-important-features-through","repo_url":"https://github.com/ascillitoe/shap","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}},{"paper_slug":"learning-important-features-through","repo_url":"https://github.com/austinbrown34/shap","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"learning-important-features-through","repo_url":"https://github.com/bips-hb/innsight","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"torch","reach":{"status":"unanswered"}},{"paper_slug":"learning-important-features-through","repo_url":"https://github.com/gablabc/shap","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}},{"paper_slug":"learning-important-features-through","repo_url":"https://github.com/miaolan-xie/shap","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}},{"paper_slug":"learning-important-features-through","repo_url":"https://github.com/pytorch/captum","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"BSD-3-Clause"}},{"paper_slug":"learning-important-features-through","repo_url":"https://github.com/saivarunr/xshap","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"learning-important-features-through","repo_url":"https://github.com/shaoshanglqy/shap-shapley","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"learning-important-features-through","repo_url":"https://github.com/shap/shap","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"learning-important-features-through","repo_url":"https://github.com/slundberg/shap","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"learning-important-features-through","repo_url":"https://github.com/vlue-c/PyTorch-Explanations","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"learning-important-features-through","repo_url":"https://github.com/vlue-c/Visual-Explanation-Methods-PyTorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"learning-important-features-through","repo_url":"https://github.com/xiaoyanLi629/ScRNA-seq-integration-by-Heterogeneous-Graph-transformer-neural-network","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"interpretable-machine-learning","task_name":"Interpretable Machine Learning"}],"methods":[{"method_slug":"interpretability","method_name":"Interpretability"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1704.02685","atlas_url":"https://app.syntology.ai/?focus=1704.02685","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1704.02685"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/pytorch/captum","reach":{"status":"ok","spdx":"BSD-3-Clause"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/adrhill/explainableai.jl","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/vlue-c/PyTorch-Explanations","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ascillitoe/shap","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/bips-hb/innsight","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/miaolan-xie/shap","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/slundberg/shap","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/austinbrown34/shap","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/shaoshanglqy/shap-shapley","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/saivarunr/xshap","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/shap/shap","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/xiaoyanLi629/ScRNA-seq-integration-by-Heterogeneous-Graph-transformer-neural-network","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/gablabc/shap","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/vlue-c/Visual-Explanation-Methods-PyTorch","reach":null}],"summary":{"ran":4,"ran_draft_wrong":3,"ran_violates":1,"unverified":7},"by_repo_kind":{"listed":{"samples":13,"ran":7,"repositories":4}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"f70e444e11ee8d3a","entry":"DeepLift","repo":"vlue-c/Visual-Explanation-Methods-PyTorch","repo_kind":"listed","path":"torchvex/contrib/deeplift.py","file_url":"https://github.com/vlue-c/Visual-Explanation-Methods-PyTorch/blob/HEAD/torchvex/contrib/deeplift.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f70e444e11ee8d3a"}},{"code_sha256_prefix":"baff092b503bcfbe","entry":"ExplanationMethod","repo":"vlue-c/Visual-Explanation-Methods-PyTorch","repo_kind":"listed","path":"torchvex/contrib/deeplift.py","file_url":"https://github.com/vlue-c/Visual-Explanation-Methods-PyTorch/blob/HEAD/torchvex/contrib/deeplift.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"baff092b503bcfbe"}},{"code_sha256_prefix":"17c52f66fb9f49e0","entry":"PyTorchDeepExplainer","repo":"austinbrown34/shap","repo_kind":"listed","path":"shap/explainers/deep/deep_pytorch.py","file_url":"https://github.com/austinbrown34/shap/blob/HEAD/shap/explainers/deep/deep_pytorch.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"17c52f66fb9f49e0"}},{"code_sha256_prefix":"315e69a6733bc503","entry":"PyTorchDeepExplainer","repo":"saivarunr/xshap","repo_kind":"listed","path":"shap/explainers/deep/deep_pytorch.py","file_url":"https://github.com/saivarunr/xshap/blob/HEAD/shap/explainers/deep/deep_pytorch.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"315e69a6733bc503"}},{"code_sha256_prefix":"174664b8521b7367","entry":"backward_walk_ops","repo":"shaoshanglqy/shap-shapley","repo_kind":"listed","path":"shap/explainers/deep/deep_tf.py","file_url":"https://github.com/shaoshanglqy/shap-shapley/blob/HEAD/shap/explainers/deep/deep_tf.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"174664b8521b7367"}},{"code_sha256_prefix":"2f372a731d99a85b","entry":"deeplift_grad","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"2f372a731d99a85b"}},{"code_sha256_prefix":"556ef4411f7dfff0","entry":"forward_walk_ops","repo":"shaoshanglqy/shap-shapley","repo_kind":"listed","path":"shap/explainers/deep/deep_tf.py","file_url":"https://github.com/shaoshanglqy/shap-shapley/blob/HEAD/shap/explainers/deep/deep_tf.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"556ef4411f7dfff0"}},{"code_sha256_prefix":"42c14f4de4f2d1ba","entry":"tensors_blocked_by_false","repo":"shaoshanglqy/shap-shapley","repo_kind":"listed","path":"shap/explainers/deep/deep_tf.py","file_url":"https://github.com/shaoshanglqy/shap-shapley/blob/HEAD/shap/explainers/deep/deep_tf.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"42c14f4de4f2d1ba"}},{"code_sha256_prefix":"6cddc3a37d231206","entry":"Explainer","repo":"shaoshanglqy/shap-shapley","repo_kind":"listed","path":"shap/explainers/deep/deep_tf.py","file_url":"https://github.com/shaoshanglqy/shap-shapley/blob/HEAD/shap/explainers/deep/deep_tf.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6cddc3a37d231206"}},{"code_sha256_prefix":"b802ce233152291a","entry":"TFDeepExplainer","repo":"shaoshanglqy/shap-shapley","repo_kind":"listed","path":"shap/explainers/deep/deep_tf.py","file_url":"https://github.com/shaoshanglqy/shap-shapley/blob/HEAD/shap/explainers/deep/deep_tf.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b802ce233152291a"}},{"code_sha256_prefix":"034debadf477812f","entry":"add_interim_values","repo":"austinbrown34/shap","repo_kind":"listed","path":"shap/explainers/deep/deep_pytorch.py","file_url":"https://github.com/austinbrown34/shap/blob/HEAD/shap/explainers/deep/deep_pytorch.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"034debadf477812f"}},{"code_sha256_prefix":"d30fb55d50e88ecc","entry":"break_dependence","repo":"shaoshanglqy/shap-shapley","repo_kind":"listed","path":"shap/explainers/deep/deep_tf.py","file_url":"https://github.com/shaoshanglqy/shap-shapley/blob/HEAD/shap/explainers/deep/deep_tf.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d30fb55d50e88ecc"}},{"code_sha256_prefix":"10adb065b8e203e6","entry":"deeplift_tensor_grad","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"10adb065b8e203e6"}},{"code_sha256_prefix":"52628a5c5ffac3aa","entry":"get_target_input","repo":"austinbrown34/shap","repo_kind":"listed","path":"shap/explainers/deep/deep_pytorch.py","file_url":"https://github.com/austinbrown34/shap/blob/HEAD/shap/explainers/deep/deep_pytorch.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"52628a5c5ffac3aa"}},{"code_sha256_prefix":"70789f676e5840c3","entry":"passthrough","repo":"shaoshanglqy/shap-shapley","repo_kind":"listed","path":"shap/explainers/deep/deep_tf.py","file_url":"https://github.com/shaoshanglqy/shap-shapley/blob/HEAD/shap/explainers/deep/deep_tf.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"70789f676e5840c3"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}