{"url":"/method/convlstm","slug":"convlstm","name":"ConvLSTM","full_name":"ConvLSTM","full_name_withheld":false,"description_markdown":"**ConvLSTM** is a type of recurrent neural network for spatio-temporal prediction that has convolutional structures in both the input-to-state and state-to-state transitions. The ConvLSTM determines the future state of a certain cell in the grid by the inputs and past states of its local neighbors. This can easily be achieved by using a [convolution](https://paperswithcode.com/method/convolution) operator in the state-to-state and input-to-state transitions (see Figure). The key equations of ConvLSTM are shown  below, where $∗$ denotes the convolution operator and $\\odot$ the Hadamard product:\r\n\r\n$$ i\\_{t} = \\sigma\\left(W\\_{xi} ∗ X\\_{t} + W\\_{hi} ∗ H\\_{t−1} + W\\_{ci} \\odot \\mathcal{C}\\_{t−1} + b\\_{i}\\right) $$\r\n\r\n$$ f\\_{t} = \\sigma\\left(W\\_{xf} ∗ X\\_{t} + W\\_{hf} ∗ H\\_{t−1} + W\\_{cf} \\odot \\mathcal{C}\\_{t−1} + b\\_{f}\\right) $$\r\n\r\n$$ \\mathcal{C}\\_{t} = f\\_{t} \\odot \\mathcal{C}\\_{t−1} + i\\_{t} \\odot \\text{tanh}\\left(W\\_{xc} ∗ X\\_{t} + W\\_{hc} ∗ \\mathcal{H}\\_{t−1} + b\\_{c}\\right) $$\r\n\r\n$$ o\\_{t} = \\sigma\\left(W\\_{xo} ∗ X\\_{t} + W\\_{ho} ∗ \\mathcal{H}\\_{t−1} + W\\_{co} \\odot \\mathcal{C}\\_{t} + b\\_{o}\\right) $$\r\n\r\n$$ \\mathcal{H}\\_{t} = o\\_{t} \\odot \\text{tanh}\\left(C\\_{t}\\right) $$\r\n\r\nIf we view the states as the hidden representations of moving objects, a ConvLSTM with a larger transitional kernel should be able to capture faster motions while one with a smaller kernel can capture slower motions. \r\n\r\nTo ensure that the states have the same number of rows and same number of columns as the inputs, padding is needed before applying the convolution operation. Here, padding of the hidden states on the boundary points can be viewed as using the state of the outside world for calculation. Usually, before the first input comes, we initialize all the states of the [LSTM](https://paperswithcode.com/method/lstm) to zero which corresponds to \"total ignorance\" of the future.","description_state":"present","introduced_year":null,"introduced_by":{"title":"Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting","paper":"/paper/convolutional-lstm-network-a-machine-learning","first_author":"Xingjian Shi","n_authors":6,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/convolutional-lstm-network-a-machine-learning"},"source":{"url":"http://arxiv.org/abs/1506.04214v2","title":"Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"Sequential","area_id":"sequential","collection":"Recurrent Neural Networks","url":"/methods/category/recurrent-neural-networks","pwc_aliases":[]}],"n_papers_tagged":145,"archive_num_papers":145,"papers_newest_first":[{"paper":null,"title":"FINN-GL: Generalized Mixed-Precision Extensions for FPGA-Accelerated LSTMs","date":"2025-06-25","arxiv_id":"2506.20810","n_code_links":0,"syntology":null},{"paper":null,"title":"ReCoGNet: Recurrent Context-Guided Network for 3D MRI Prostate Segmentation","date":"2025-06-24","arxiv_id":"2506.19687","n_code_links":0,"syntology":null},{"paper":null,"title":"Residual Connection-Enhanced ConvLSTM for Lithium Dendrite Growth Prediction","date":"2025-06-21","arxiv_id":"2506.17756","n_code_links":0,"syntology":null},{"paper":null,"title":"Deep Learning Weather Models for Subregional Ocean Forecasting: A Case Study on the Canary Current Upwelling System","date":"2025-05-30","arxiv_id":"2505.24429","n_code_links":0,"syntology":null},{"paper":null,"title":"Convolutional Long Short-Term Memory Neural Networks Based Numerical Simulation of Flow Field","date":"2025-05-21","arxiv_id":"2505.15533","n_code_links":0,"syntology":null},{"paper":null,"title":"CTP: A hybrid CNN-Transformer-PINN model for ocean front forecasting","date":"2025-05-16","arxiv_id":"2505.10894","n_code_links":0,"syntology":null},{"paper":"/paper/domain-knowledge-integrated-cnn-xlstm-xatt","title":"Domain Knowledge Integrated CNN-xLSTM-xAtt Network with Multi Stream Feature Fusion for Cuffless Blood Pressure Estimation from Photoplethysmography Signals","date":"2025-05-13","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":null,"title":"Global Climate Model Bias Correction Using Deep Learning","date":"2025-04-27","arxiv_id":"2504.19145","n_code_links":0,"syntology":null},{"paper":null,"title":"How to systematically develop an effective AI-based bias correction model?","date":"2025-04-21","arxiv_id":"2504.15322","n_code_links":0,"syntology":null},{"paper":null,"title":"Advancing Video Anomaly Detection: A Bi-Directional Hybrid Framework for Enhanced Single- and Multi-Task Approaches","date":"2025-04-20","arxiv_id":"2504.14753","n_code_links":0,"syntology":null},{"paper":null,"title":"Exploring FMCW Radars and Feature Maps for Activity Recognition: A Benchmark Study","date":"2025-03-07","arxiv_id":"2503.05629","n_code_links":0,"syntology":null},{"paper":null,"title":"Regional climate projections using a deep-learning-based model-ranking and downscaling framework: Application to European climate zones","date":"2025-02-27","arxiv_id":"2502.20132","n_code_links":0,"syntology":null},{"paper":"/paper/iot-based-real-time-medical-related-human","title":"IoT-Based Real-Time Medical-Related Human Activity Recognition Using Skeletons and Multi-Stage Deep Learning for Healthcare","date":"2025-01-13","arxiv_id":"2501.07039","n_code_links":1,"syntology":null},{"paper":null,"title":"TopoFormer: Integrating Transformers and ConvLSTMs for Coastal Topography Prediction","date":"2025-01-11","arxiv_id":"2501.06494","n_code_links":0,"syntology":null},{"paper":null,"title":"An End-to-End Two-Stream Network Based on RGB Flow and Representation Flow for Human Action Recognition","date":"2024-11-27","arxiv_id":"2411.18002","n_code_links":0,"syntology":null},{"paper":null,"title":"Video to Video Generative Adversarial Network for Few-shot Learning Based on Policy Gradient","date":"2024-10-28","arxiv_id":"2410.20657","n_code_links":0,"syntology":null},{"paper":"/paper/predformer-transformers-are-effective-spatial","title":"Video Prediction Transformers without Recurrence or Convolution","date":"2024-10-07","arxiv_id":"2410.04733","n_code_links":1,"syntology":null},{"paper":"/paper/dynamical-system-prediction-from-sparse","title":"Dynamical system prediction from sparse observations using deep neural networks with Voronoi tessellation and physics constraint","date":"2024-08-31","arxiv_id":"2409.00458","n_code_links":1,"syntology":null},{"paper":null,"title":"An Improved CovidConvLSTM model for pneumonia-COVID-19 detection and classification","date":"2024-08-21","arxiv_id":"2408.11507","n_code_links":0,"syntology":null},{"paper":"/paper/fate-focal-modulated-attention-encoder-for","title":"FATE: Focal-modulated Attention Encoder for Temperature Prediction","date":"2024-08-21","arxiv_id":"2408.11336","n_code_links":1,"syntology":null},{"paper":"/paper/comparing-and-contrasting-deep-learning","title":"Comparing and Contrasting Deep Learning Weather Prediction Backbones on Navier-Stokes and Atmospheric Dynamics","date":"2024-07-19","arxiv_id":"2407.14129","n_code_links":1,"syntology":{"ran":1,"of":2,"unverified":1,"pointer_only":0}},{"paper":null,"title":"Global Spatial-Temporal Information-based Residual ConvLSTM for Video Space-Time Super-Resolution","date":"2024-07-11","arxiv_id":"2407.08466","n_code_links":0,"syntology":null},{"paper":null,"title":"Deformable Feature Alignment and Refinement for Moving Infrared Dim-small Target Detection","date":"2024-07-10","arxiv_id":"2407.07289","n_code_links":0,"syntology":null},{"paper":null,"title":"Deep Learning for Prediction and Classifying the Dynamical behaviour of Piecewise Smooth Maps","date":"2024-06-24","arxiv_id":"2406.17001","n_code_links":0,"syntology":null},{"paper":"/paper/exploring-the-efficacy-of-a-hybrid-approach","title":"Generalization capabilities and robustness of hybrid models grounded in physics compared to purely deep learning models","date":"2024-04-27","arxiv_id":"2404.17884","n_code_links":1,"syntology":null},{"paper":"/paper/vestibular-schwannoma-growth-prediction-from","title":"Vestibular schwannoma growth prediction from longitudinal MRI by time conditioned neural fields","date":"2024-04-03","arxiv_id":"2404.02614","n_code_links":1,"syntology":null},{"paper":null,"title":"Predicting machine failures from multivariate time series: an industrial case study","date":"2024-02-27","arxiv_id":"2402.17804","n_code_links":0,"syntology":null},{"paper":"/paper/polypnextlstm-a-lightweight-and-fast-polyp","title":"PolypNextLSTM: A lightweight and fast polyp video segmentation network using ConvNext and ConvLSTM","date":"2024-02-18","arxiv_id":"2402.11585","n_code_links":1,"syntology":null},{"paper":"/paper/short-term-precipitation-forecasting-in-the","title":"Short-term Precipitation Forecasting in The Netherlands: An Application of Convolutional LSTM neural networks to weather radar data","date":"2023-12-02","arxiv_id":"2312.01197","n_code_links":1,"syntology":null},{"paper":null,"title":"Forecasting Post-Wildfire Vegetation Recovery in California using a Convolutional Long Short-Term Memory Tensor Regression Network","date":"2023-11-04","arxiv_id":"2311.02492","n_code_links":0,"syntology":null}],"papers_shown":30,"tasks":[{"task":"/task/prediction","name":"Prediction","papers":16},{"task":"/task/video-prediction","name":"Video Prediction","papers":15},{"task":"/task/deep-learning","name":"Deep Learning","papers":13},{"task":"/task/semantic-segmentation","name":"Semantic Segmentation","papers":11},{"task":"/task/time-series-1","name":"Time Series","papers":11},{"task":"/task/decoder","name":"Decoder","papers":10},{"task":"/task/segmentation","name":"Segmentation","papers":9},{"task":"/task/time-series","name":"Time Series Analysis","papers":9},{"task":"/task/image-segmentation","name":"Image Segmentation","papers":8},{"task":"/task/optical-flow-estimation","name":"Optical Flow Estimation","papers":8},{"task":"/task/activity-recognition","name":"Activity Recognition","papers":7},{"task":"/task/super-resolution","name":"Super-Resolution","papers":6},{"task":"/task/weather-forecasting","name":"Weather Forecasting","papers":6},{"task":"/task/object-detection","name":"Object Detection","papers":5},{"task":"/task/video-super-resolution","name":"Video Super-Resolution","papers":5},{"task":"/task/object-detection-1","name":"object-detection","papers":5},{"task":"/task/action-recognition-in-videos","name":"Action Recognition","papers":4},{"task":"/task/anomaly-detection","name":"Anomaly Detection","papers":4},{"task":"/task/human-activity-recognition","name":"Human Activity Recognition","papers":4},{"task":"/task/space-time-video-super-resolution","name":"Space-time Video Super-resolution","papers":4}],"tasks_shown":20,"n_tasks":170,"usage_by_year":[{"year":"2015","papers":1},{"year":"2017","papers":3},{"year":"2018","papers":9},{"year":"2019","papers":20},{"year":"2020","papers":26},{"year":"2021","papers":20},{"year":"2022","papers":19},{"year":"2023","papers":19},{"year":"2024","papers":14},{"year":"2025","papers":14}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/convlstm"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}