{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/hippo-recurrent-memory-with-optimal","title":"HiPPO: Recurrent Memory with Optimal Polynomial Projections","arxiv_id":"2008.07669","date":"2020-08-17","proceeding":"NeurIPS 2020 12","authors":["Albert Gu","Tri Dao","Stefano Ermon","Atri Rudra","Christopher Re"],"abstract":"A central problem in learning from sequential data is representing cumulative history in an incremental fashion as more data is processed. We introduce a general framework (HiPPO) for the online compression of continuous signals and discrete time series by projection onto polynomial bases. Given a measure that specifies the importance of each time step in the past, HiPPO produces an optimal solution to a natural online function approximation problem. As special cases, our framework yields a short derivation of the recent Legendre Memory Unit (LMU) from first principles, and generalizes the ubiquitous gating mechanism of recurrent neural networks such as GRUs. This formal framework yields a new memory update mechanism (HiPPO-LegS) that scales through time to remember all history, avoiding priors on the timescale. HiPPO-LegS enjoys the theoretical benefits of timescale robustness, fast updates, and bounded gradients. By incorporating the memory dynamics into recurrent neural networks, HiPPO RNNs can empirically capture complex temporal dependencies. On the benchmark permuted MNIST dataset, HiPPO-LegS sets a new state-of-the-art accuracy of 98.3%. Finally, on a novel trajectory classification task testing robustness to out-of-distribution timescales and missing data, HiPPO-LegS outperforms RNN and neural ODE baselines by 25-40% accuracy.","url_abs":"https://arxiv.org/abs/2008.07669v2","url_pdf":"https://arxiv.org/pdf/2008.07669v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"hippo-recurrent-memory-with-optimal","repo_url":"https://github.com/HazyResearch/hippo-code","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"hippo-recurrent-memory-with-optimal","repo_url":"https://github.com/ag1988/dss","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":null,"task_name":"Permuted-MNIST"},{"task_slug":"sequential-image-classification","task_name":"Sequential Image Classification"},{"task_slug":"time-series-1","task_name":"Time Series"},{"task_slug":"time-series","task_name":"Time Series Analysis"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/sequential-image-classification-on-sequential","task":"Sequential Image Classification","dataset":"Sequential MNIST","model":"HiPPO-LegS","rank_in_archive_order":8,"of":30,"metrics":{"Permuted Accuracy":"98.3%"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2008.07669","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2008.07669"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ag1988/dss","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/HazyResearch/hippo-code","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_draft_wrong":3,"ran":3,"unverified":6},"by_repo_kind":{"official":{"samples":10,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"89b10a8b26bbb7ed","entry":"Activation","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"89b10a8b26bbb7ed"}},{"code_sha256_prefix":"920218357ef96393","entry":"bitreversal_permutation","repo":"HazyResearch/hippo-code","repo_kind":"official","path":"model/unroll.py","file_url":"https://github.com/HazyResearch/hippo-code/blob/HEAD/model/unroll.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"920218357ef96393"}},{"code_sha256_prefix":"1c7d80fa5881a795","entry":"bitreversal_po2","repo":"HazyResearch/hippo-code","repo_kind":"official","path":"model/unroll.py","file_url":"https://github.com/HazyResearch/hippo-code/blob/HEAD/model/unroll.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1c7d80fa5881a795"}},{"code_sha256_prefix":"f3449af8fae1d130","entry":"get_initializer","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"f3449af8fae1d130"}},{"code_sha256_prefix":"3de2f7ab3e3dc431","entry":"shift_up","repo":"HazyResearch/hippo-code","repo_kind":"official","path":"model/unroll.py","file_url":"https://github.com/HazyResearch/hippo-code/blob/HEAD/model/unroll.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3de2f7ab3e3dc431"}},{"code_sha256_prefix":"82ea253c64044e55","entry":"transition","repo":"HazyResearch/hippo-code","repo_kind":"official","path":"model/op.py","file_url":"https://github.com/HazyResearch/hippo-code/blob/HEAD/model/op.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"82ea253c64044e55"}},{"code_sha256_prefix":"ba910cfe698ad603","entry":"Linear_","repo":"HazyResearch/hippo-code","repo_kind":"official","path":"model/components.py","file_url":"https://github.com/HazyResearch/hippo-code/blob/HEAD/model/components.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ba910cfe698ad603"}},{"code_sha256_prefix":"b24ed7ecbd6249ab","entry":"apply_tuple","repo":"HazyResearch/hippo-code","repo_kind":"official","path":"model/rnn.py","file_url":"https://github.com/HazyResearch/hippo-code/blob/HEAD/model/rnn.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b24ed7ecbd6249ab"}},{"code_sha256_prefix":"d25cbd25aadd7fea","entry":"concat_tuple","repo":"HazyResearch/hippo-code","repo_kind":"official","path":"model/rnn.py","file_url":"https://github.com/HazyResearch/hippo-code/blob/HEAD/model/rnn.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d25cbd25aadd7fea"}},{"code_sha256_prefix":"261aab6fe51ea01d","entry":"get_initializer","repo":"HazyResearch/hippo-code","repo_kind":"official","path":"model/components.py","file_url":"https://github.com/HazyResearch/hippo-code/blob/HEAD/model/components.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"261aab6fe51ea01d"}},{"code_sha256_prefix":"662bbcc5aee6ea80","entry":"pl_train","repo":"HazyResearch/hippo-code","repo_kind":"official","path":"pl_runner.py","file_url":"https://github.com/HazyResearch/hippo-code/blob/HEAD/pl_runner.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"662bbcc5aee6ea80"}},{"code_sha256_prefix":"46b2822b9e5411e0","entry":"transition","repo":"HazyResearch/hippo-code","repo_kind":"official","path":"tensorflow/hippo.py","file_url":"https://github.com/HazyResearch/hippo-code/blob/HEAD/tensorflow/hippo.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"46b2822b9e5411e0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}