{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/beyond-word-importance-contextual","title":"Beyond Word Importance: Contextual Decomposition to Extract Interactions from LSTMs","arxiv_id":"1801.05453","date":"2018-01-16","proceeding":"ICLR 2018 1","authors":["W. James Murdoch","Peter J. Liu","Bin Yu"],"abstract":"The driving force behind the recent success of LSTMs has been their ability\nto learn complex and non-linear relationships. Consequently, our inability to\ndescribe these relationships has led to LSTMs being characterized as black\nboxes. To this end, we introduce contextual decomposition (CD), an\ninterpretation algorithm for analysing individual predictions made by standard\nLSTMs, without any changes to the underlying model. By decomposing the output\nof a LSTM, CD captures the contributions of combinations of words or variables\nto the final prediction of an LSTM. On the task of sentiment analysis with the\nYelp and SST data sets, we show that CD is able to reliably identify words and\nphrases of contrasting sentiment, and how they are combined to yield the LSTM's\nfinal prediction. Using the phrase-level labels in SST, we also demonstrate\nthat CD is able to successfully extract positive and negative negations from an\nLSTM, something which has not previously been done.","url_abs":"http://arxiv.org/abs/1801.05453v2","url_pdf":"http://arxiv.org/pdf/1801.05453v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"beyond-word-importance-contextual","repo_url":"https://github.com/jamie-murdoch/ContextualDecomposition","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"beyond-word-importance-contextual","repo_url":"https://github.com/optum/long-medical-document-lms","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"beyond-word-importance-contextual","repo_url":"https://github.com/suyash/ContextualDecomposition","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"beyond-word-importance-contextual","repo_url":"https://github.com/csinva/hierarchical-dnn-interpretations","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"sentiment-analysis","task_name":"Sentiment Analysis"}],"methods":[{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1801.05453","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1801.05453"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/suyash/ContextualDecomposition","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/optum/long-medical-document-lms","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/csinva/hierarchical-dnn-interpretations","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jamie-murdoch/ContextualDecomposition","reach":{"status":"unanswered"}}],"summary":{"ran_draft_wrong":1},"by_repo_kind":{"listed":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"61afc4e6b34d81cf","entry":"act_decomposition","repo":"suyash/ContextualDecomposition","repo_kind":"listed","path":"cd/cd.py","file_url":"https://github.com/suyash/ContextualDecomposition/blob/HEAD/cd/cd.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"61afc4e6b34d81cf"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}