{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/accurate-grid-keypoint-learning-for-efficient","title":"Accurate Grid Keypoint Learning for Efficient Video Prediction","arxiv_id":"2107.13170","date":"2021-07-28","proceeding":null,"authors":["Xiaojie Gao","Yueming Jin","Qi Dou","Chi-Wing Fu","Pheng-Ann Heng"],"abstract":"Video prediction methods generally consume substantial computing resources in training and deployment, among which keypoint-based approaches show promising improvement in efficiency by simplifying dense image prediction to light keypoint prediction. However, keypoint locations are often modeled only as continuous coordinates, so noise from semantically insignificant deviations in videos easily disrupt learning stability, leading to inaccurate keypoint modeling. In this paper, we design a new grid keypoint learning framework, aiming at a robust and explainable intermediate keypoint representation for long-term efficient video prediction. We have two major technical contributions. First, we detect keypoints by jumping among candidate locations in our raised grid space and formulate a condensation loss to encourage meaningful keypoints with strong representative capability. Second, we introduce a 2D binary map to represent the detected grid keypoints and then suggest propagating keypoint locations with stochasticity by selecting entries in the discrete grid space, thus preserving the spatial structure of keypoints in the longterm horizon for better future frame generation. Extensive experiments verify that our method outperforms the state-ofthe-art stochastic video prediction methods while saves more than 98% of computing resources. We also demonstrate our method on a robotic-assisted surgery dataset with promising results. Our code is available at https://github.com/xjgaocs/Grid-Keypoint-Learning.","url_abs":"https://arxiv.org/abs/2107.13170v1","url_pdf":"https://arxiv.org/pdf/2107.13170v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"accurate-grid-keypoint-learning-for-efficient","repo_url":"https://github.com/xjgaocs/Grid-Keypoint-Learning","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"prediction","task_name":"Prediction"},{"task_slug":"video-prediction","task_name":"Video Prediction"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/video-prediction-on-kth","task":"Video Prediction","dataset":"KTH","model":"Grid-keypoints","rank_in_archive_order":1,"of":31,"metrics":{"Cond":"10","FVD":"144.2","LPIPS":"0.092","PSNR":"27.11","Params (M)":"2.0","Pred":"40","SSIM":"0.837","Train":"10"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2107.13170","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2107.13170"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/xjgaocs/Grid-Keypoint-Learning","reach":null}],"summary":{"ran_fixture":1,"ran_honours":1},"by_repo_kind":{"official":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"8770e6bc1c98f605","entry":"CalDelta_xy","repo":"xjgaocs/Grid-Keypoint-Learning","repo_kind":"official","path":"train_all_AE_BN3_kth_1_2.py","file_url":"https://github.com/xjgaocs/Grid-Keypoint-Learning/blob/HEAD/train_all_AE_BN3_kth_1_2.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8770e6bc1c98f605"}},{"code_sha256_prefix":"b29795f4755b91e1","entry":"kl_criterion","repo":"xjgaocs/Grid-Keypoint-Learning","repo_kind":"official","path":"train_all_AE_BN3_kth_1_2.py","file_url":"https://github.com/xjgaocs/Grid-Keypoint-Learning/blob/HEAD/train_all_AE_BN3_kth_1_2.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b29795f4755b91e1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}