{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/how-recurrent-networks-implement-contextual","title":"How recurrent networks implement contextual processing in sentiment analysis","arxiv_id":"2004.08013","date":"2020-04-17","proceeding":"ICML 2020 1","authors":["Niru Maheswaranathan","David Sussillo"],"abstract":"Neural networks have a remarkable capacity for contextual processing--using recent or nearby inputs to modify processing of current input. For example, in natural language, contextual processing is necessary to correctly interpret negation (e.g. phrases such as \"not bad\"). However, our ability to understand how networks process context is limited. Here, we propose general methods for reverse engineering recurrent neural networks (RNNs) to identify and elucidate contextual processing. We apply these methods to understand RNNs trained on sentiment classification. This analysis reveals inputs that induce contextual effects, quantifies the strength and timescale of these effects, and identifies sets of these inputs with similar properties. Additionally, we analyze contextual effects related to differential processing of the beginning and end of documents. Using the insights learned from the RNNs we improve baseline Bag-of-Words models with simple extensions that incorporate contextual modification, recovering greater than 90% of the RNN's performance increase over the baseline. This work yields a new understanding of how RNNs process contextual information, and provides tools that should provide similar insight more broadly.","url_abs":"https://arxiv.org/abs/2004.08013v1","url_pdf":"https://arxiv.org/pdf/2004.08013v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"how-recurrent-networks-implement-contextual","repo_url":"https://github.com/google-research/computation-thru-dynamics","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"negation","task_name":"Negation"},{"task_slug":"sentiment-analysis","task_name":"Sentiment Analysis"},{"task_slug":"sentiment-classification","task_name":"Sentiment Classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2004.08013","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2004.08013"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/google-research/computation-thru-dynamics","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":19},"by_repo_kind":{"listed":{"samples":19,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"aced411c40f5de2f","entry":"affine","repo":"google-research/computation-thru-dynamics","repo_kind":"listed","path":"integrator_rnn_tutorial/rnn.py","file_url":"https://github.com/google-research/computation-thru-dynamics/blob/HEAD/integrator_rnn_tutorial/rnn.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"aced411c40f5de2f"}},{"code_sha256_prefix":"896edacf34bb8bd8","entry":"diag_gaussian_log_likelihood","repo":"google-research/computation-thru-dynamics","repo_kind":"listed","path":"lfads_tutorial/distributions.py","file_url":"https://github.com/google-research/computation-thru-dynamics/blob/HEAD/lfads_tutorial/distributions.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"896edacf34bb8bd8"}},{"code_sha256_prefix":"0b1d13b86e3d6a9e","entry":"diag_gaussian_sample","repo":"google-research/computation-thru-dynamics","repo_kind":"listed","path":"lfads_tutorial/distributions.py","file_url":"https://github.com/google-research/computation-thru-dynamics/blob/HEAD/lfads_tutorial/distributions.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0b1d13b86e3d6a9e"}},{"code_sha256_prefix":"34a73dab2ffc9aa7","entry":"get_fp_loss_fun","repo":"google-research/computation-thru-dynamics","repo_kind":"listed","path":"fixed_point_finder/fixed_points.py","file_url":"https://github.com/google-research/computation-thru-dynamics/blob/HEAD/fixed_point_finder/fixed_points.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"34a73dab2ffc9aa7"}},{"code_sha256_prefix":"2cc94ae064dc4dd7","entry":"get_kl_warmup_fun","repo":"google-research/computation-thru-dynamics","repo_kind":"listed","path":"lfads_tutorial/optimize.py","file_url":"https://github.com/google-research/computation-thru-dynamics/blob/HEAD/lfads_tutorial/optimize.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2cc94ae064dc4dd7"}},{"code_sha256_prefix":"c873384736c6ede9","entry":"get_total_fp_loss_fun","repo":"google-research/computation-thru-dynamics","repo_kind":"listed","path":"fixed_point_finder/fixed_points.py","file_url":"https://github.com/google-research/computation-thru-dynamics/blob/HEAD/fixed_point_finder/fixed_points.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c873384736c6ede9"}},{"code_sha256_prefix":"3976c1f7c4e6dab7","entry":"gru","repo":"google-research/computation-thru-dynamics","repo_kind":"listed","path":"fixed_point_finder/rnn.py","file_url":"https://github.com/google-research/computation-thru-dynamics/blob/HEAD/fixed_point_finder/rnn.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3976c1f7c4e6dab7"}},{"code_sha256_prefix":"295c6b5c69a5cfa0","entry":"keygen","repo":"google-research/computation-thru-dynamics","repo_kind":"listed","path":"fixed_point_finder/utils.py","file_url":"https://github.com/google-research/computation-thru-dynamics/blob/HEAD/fixed_point_finder/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"295c6b5c69a5cfa0"}},{"code_sha256_prefix":"3fad852c8d126922","entry":"merge_losses_dicts","repo":"google-research/computation-thru-dynamics","repo_kind":"listed","path":"fixed_point_finder/utils.py","file_url":"https://github.com/google-research/computation-thru-dynamics/blob/HEAD/fixed_point_finder/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3fad852c8d126922"}},{"code_sha256_prefix":"23408171ea05d4c3","entry":"plot_data_example","repo":"google-research/computation-thru-dynamics","repo_kind":"listed","path":"lfads_tutorial/plotting.py","file_url":"https://github.com/google-research/computation-thru-dynamics/blob/HEAD/lfads_tutorial/plotting.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"23408171ea05d4c3"}},{"code_sha256_prefix":"5f40340057d4af85","entry":"plot_data_pca","repo":"google-research/computation-thru-dynamics","repo_kind":"listed","path":"lfads_tutorial/plotting.py","file_url":"https://github.com/google-research/computation-thru-dynamics/blob/HEAD/lfads_tutorial/plotting.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5f40340057d4af85"}},{"code_sha256_prefix":"ae64d899719d4eb9","entry":"plot_data_stats","repo":"google-research/computation-thru-dynamics","repo_kind":"listed","path":"lfads_tutorial/plotting.py","file_url":"https://github.com/google-research/computation-thru-dynamics/blob/HEAD/lfads_tutorial/plotting.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ae64d899719d4eb9"}},{"code_sha256_prefix":"a80bb6e1b393fe43","entry":"poisson_log_likelihood","repo":"google-research/computation-thru-dynamics","repo_kind":"listed","path":"lfads_tutorial/distributions.py","file_url":"https://github.com/google-research/computation-thru-dynamics/blob/HEAD/lfads_tutorial/distributions.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a80bb6e1b393fe43"}},{"code_sha256_prefix":"670f241a0a278e35","entry":"read_file","repo":"google-research/computation-thru-dynamics","repo_kind":"listed","path":"fixed_point_finder/utils.py","file_url":"https://github.com/google-research/computation-thru-dynamics/blob/HEAD/fixed_point_finder/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"670f241a0a278e35"}},{"code_sha256_prefix":"deee5acedc65539b","entry":"sigmoid","repo":"google-research/computation-thru-dynamics","repo_kind":"listed","path":"fixed_point_finder/rnn.py","file_url":"https://github.com/google-research/computation-thru-dynamics/blob/HEAD/fixed_point_finder/rnn.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"deee5acedc65539b"}},{"code_sha256_prefix":"48f60e785be2e858","entry":"sigmoid","repo":"google-research/computation-thru-dynamics","repo_kind":"listed","path":"lfads_tutorial/lfads.py","file_url":"https://github.com/google-research/computation-thru-dynamics/blob/HEAD/lfads_tutorial/lfads.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"48f60e785be2e858"}},{"code_sha256_prefix":"846c9d4c4a3c7b6c","entry":"spikify_data","repo":"google-research/computation-thru-dynamics","repo_kind":"listed","path":"lfads_tutorial/utils.py","file_url":"https://github.com/google-research/computation-thru-dynamics/blob/HEAD/lfads_tutorial/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"846c9d4c4a3c7b6c"}},{"code_sha256_prefix":"1f5c168fff169bd8","entry":"split_data","repo":"google-research/computation-thru-dynamics","repo_kind":"listed","path":"lfads_tutorial/utils.py","file_url":"https://github.com/google-research/computation-thru-dynamics/blob/HEAD/lfads_tutorial/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1f5c168fff169bd8"}},{"code_sha256_prefix":"61a25573238a87a1","entry":"vrnn","repo":"google-research/computation-thru-dynamics","repo_kind":"listed","path":"integrator_rnn_tutorial/rnn.py","file_url":"https://github.com/google-research/computation-thru-dynamics/blob/HEAD/integrator_rnn_tutorial/rnn.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"61a25573238a87a1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}