{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/continuous-online-sequence-learning-with-an","title":"Continuous online sequence learning with an unsupervised neural network model","arxiv_id":"1512.05463","date":"2015-12-17","proceeding":null,"authors":["Yuwei Cui","Subutai Ahmad","Jeff Hawkins"],"abstract":"The ability to recognize and predict temporal sequences of sensory inputs is\nvital for survival in natural environments. Based on many known properties of\ncortical neurons, hierarchical temporal memory (HTM) sequence memory is\nrecently proposed as a theoretical framework for sequence learning in the\ncortex. In this paper, we analyze properties of HTM sequence memory and apply\nit to sequence learning and prediction problems with streaming data. We show\nthe model is able to continuously learn a large number of variable-order\ntemporal sequences using an unsupervised Hebbian-like learning rule. The sparse\ntemporal codes formed by the model can robustly handle branching temporal\nsequences by maintaining multiple predictions until there is sufficient\ndisambiguating evidence. We compare the HTM sequence memory with other sequence\nlearning algorithms, including statistical methods: autoregressive integrated\nmoving average (ARIMA), feedforward neural networks: online sequential extreme\nlearning machine (ELM), and recurrent neural networks: long short-term memory\n(LSTM) and echo-state networks (ESN), on sequence prediction problems with both\nartificial and real-world data. The HTM model achieves comparable accuracy to\nother state-of-the-art algorithms. The model also exhibits properties that are\ncritical for sequence learning, including continuous online learning, the\nability to handle multiple predictions and branching sequences with high order\nstatistics, robustness to sensor noise and fault tolerance, and good\nperformance without task-specific hyper- parameters tuning. Therefore the HTM\nsequence memory not only advances our understanding of how the brain may solve\nthe sequence learning problem, but is also applicable to a wide range of\nreal-world problems such as discrete and continuous sequence prediction,\nanomaly detection, and sequence classification.","url_abs":"http://arxiv.org/abs/1512.05463v2","url_pdf":"http://arxiv.org/pdf/1512.05463v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"continuous-online-sequence-learning-with-an","repo_url":"https://github.com/shivam-131/CLA","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"anomaly-detection","task_name":"Anomaly Detection"},{"task_slug":"temporal-sequences","task_name":"Temporal Sequences"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}