{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/video-prediction-recalling-long-term-motion","title":"Video Prediction Recalling Long-term Motion Context via Memory Alignment Learning","arxiv_id":"2104.00924","date":"2021-04-02","proceeding":"CVPR 2021 1","authors":["Sangmin Lee","Hak Gu Kim","Dae Hwi Choi","Hyung-Il Kim","Yong Man Ro"],"abstract":"Our work addresses long-term motion context issues for predicting future frames. To predict the future precisely, it is required to capture which long-term motion context (e.g., walking or running) the input motion (e.g., leg movement) belongs to. The bottlenecks arising when dealing with the long-term motion context are: (i) how to predict the long-term motion context naturally matching input sequences with limited dynamics, (ii) how to predict the long-term motion context with high-dimensionality (e.g., complex motion). To address the issues, we propose novel motion context-aware video prediction. To solve the bottleneck (i), we introduce a long-term motion context memory (LMC-Memory) with memory alignment learning. The proposed memory alignment learning enables to store long-term motion contexts into the memory and to match them with sequences including limited dynamics. As a result, the long-term context can be recalled from the limited input sequence. In addition, to resolve the bottleneck (ii), we propose memory query decomposition to store local motion context (i.e., low-dimensional dynamics) and recall the suitable local context for each local part of the input individually. It enables to boost the alignment effects of the memory. Experimental results show that the proposed method outperforms other sophisticated RNN-based methods, especially in long-term condition. Further, we validate the effectiveness of the proposed network designs by conducting ablation studies and memory feature analysis. The source code of this work is available.","url_abs":"https://arxiv.org/abs/2104.00924v1","url_pdf":"https://arxiv.org/pdf/2104.00924v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"video-prediction-recalling-long-term-motion","repo_url":"https://github.com/sangmin-git/LMC-Memory","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"video-prediction","task_name":"Video Prediction"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/video-prediction-on-kth","task":"Video Prediction","dataset":"KTH","model":"LMC","rank_in_archive_order":16,"of":31,"metrics":{"Cond":"10","LPIPS":"159.8","PSNR":"27.5","Pred":"40","SSIM":"0.879"},"uses_additional_data":false},{"leaderboard":"/sota/video-prediction-on-moving-mnist","task":"Video Prediction","dataset":"Moving MNIST","model":"LMC","rank_in_archive_order":24,"of":31,"metrics":{"LPIPS":"0.047","MSE":"41.5","SSIM":"0.924"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2104.00924","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2104.00924"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sangmin-git/LMC-Memory","reach":null}],"summary":{"unverified":1},"by_repo_kind":{"official":{"samples":1,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"dded2f1ec50c63b0","entry":"Memory","repo":"sangmin-git/LMC-Memory","repo_kind":"official","path":"model.py","file_url":"https://github.com/sangmin-git/LMC-Memory/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"dded2f1ec50c63b0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}