{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/vfimamba-video-frame-interpolation-with-state","title":"VFIMamba: Video Frame Interpolation with State Space Models","arxiv_id":"2407.02315","date":"2024-07-02","proceeding":null,"authors":["Guozhen Zhang","Chunxu Liu","Yutao Cui","Xiaotong Zhao","Kai Ma","LiMin Wang"],"abstract":"Inter-frame modeling is pivotal in generating intermediate frames for video frame interpolation (VFI). Current approaches predominantly rely on convolution or attention-based models, which often either lack sufficient receptive fields or entail significant computational overheads. Recently, Selective State Space Models (S6) have emerged, tailored specifically for long sequence modeling, offering both linear complexity and data-dependent modeling capabilities. In this paper, we propose VFIMamba, a novel frame interpolation method for efficient and dynamic inter-frame modeling by harnessing the S6 model. Our approach introduces the Mixed-SSM Block (MSB), which initially rearranges tokens from adjacent frames in an interleaved fashion and subsequently applies multi-directional S6 modeling. This design facilitates the efficient transmission of information across frames while upholding linear complexity. Furthermore, we introduce a novel curriculum learning strategy that progressively cultivates proficiency in modeling inter-frame dynamics across varying motion magnitudes, fully unleashing the potential of the S6 model. Experimental findings showcase that our method attains state-of-the-art performance across diverse benchmarks, particularly excelling in high-resolution scenarios. In particular, on the X-TEST dataset, VFIMamba demonstrates a noteworthy improvement of 0.80 dB for 4K frames and 0.96 dB for 2K frames.","url_abs":"https://arxiv.org/abs/2407.02315v2","url_pdf":"https://arxiv.org/pdf/2407.02315v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"vfimamba-video-frame-interpolation-with-state","repo_url":"https://github.com/mcg-nju/vfimamba","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"2k","task_name":"2k"},{"task_slug":"4k","task_name":"4k"},{"task_slug":"state-space-models","task_name":"State Space Models"},{"task_slug":"video-frame-interpolation","task_name":"Video Frame Interpolation"}],"methods":[{"method_slug":"convolution","method_name":"Convolution"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/video-frame-interpolation-on-snu-film-easy","task":"Video Frame Interpolation","dataset":"SNU-FILM (easy)","model":"VFIMamba","rank_in_archive_order":2,"of":8,"metrics":{"PSNR":"40.51","SSIM":"0.9912"},"uses_additional_data":false},{"leaderboard":"/sota/video-frame-interpolation-on-snu-film-extreme","task":"Video Frame Interpolation","dataset":"SNU-FILM (extreme)","model":"VFIMamba","rank_in_archive_order":3,"of":8,"metrics":{"PSNR":"25.79","SSIM":"0.8682"},"uses_additional_data":false},{"leaderboard":"/sota/video-frame-interpolation-on-snu-film-hard","task":"Video Frame Interpolation","dataset":"SNU-FILM (hard)","model":"VFIMamba","rank_in_archive_order":3,"of":8,"metrics":{"PSNR":"30.99","SSIM":"0.9401"},"uses_additional_data":false},{"leaderboard":"/sota/video-frame-interpolation-on-snu-film-medium","task":"Video Frame Interpolation","dataset":"SNU-FILM (medium)","model":"VFIMamba","rank_in_archive_order":3,"of":8,"metrics":{"PSNR":"36.40","SSIM":"0.9805"},"uses_additional_data":false},{"leaderboard":"/sota/video-frame-interpolation-on-ucf101-1","task":"Video Frame Interpolation","dataset":"UCF101","model":"VFIMamba","rank_in_archive_order":3,"of":19,"metrics":{"PSNR":"35.45","SSIM":"0.9702"},"uses_additional_data":false},{"leaderboard":"/sota/video-frame-interpolation-on-x4k1000fps","task":"Video Frame Interpolation","dataset":"X4K1000FPS","model":"VFIMamba","rank_in_archive_order":3,"of":20,"metrics":{"PSNR":"32.15","SSIM":"0.9246"},"uses_additional_data":false},{"leaderboard":"/sota/video-frame-interpolation-on-x4k1000fps-2k","task":"Video Frame Interpolation","dataset":"X4K1000FPS-2K","model":"VFIMamba","rank_in_archive_order":1,"of":4,"metrics":{"PSNR":"33.33","SSIM":"0.9362"},"uses_additional_data":false},{"leaderboard":"/sota/video-frame-interpolation-on-xiph-2k","task":"Video Frame Interpolation","dataset":"Xiph-2K","model":"VFIMamba","rank_in_archive_order":1,"of":4,"metrics":{"PSNR":"37.13","SSIM":"0.9451"},"uses_additional_data":false},{"leaderboard":"/sota/video-frame-interpolation-on-xiph-4k-1","task":"Video Frame Interpolation","dataset":"Xiph-4k","model":"VFIMamba","rank_in_archive_order":2,"of":3,"metrics":{"PSNR":"34.62","SSIM":"0.9059"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2407.02315","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2407.02315"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mcg-nju/vfimamba","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_draft_wrong":1,"ran":7},"by_repo_kind":{"official":{"samples":8,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"1d13034a9dbf1c2a","entry":"conv","repo":"mcg-nju/vfimamba","repo_kind":"official","path":"model/feature_extractor.py","file_url":"https://github.com/mcg-nju/vfimamba/blob/HEAD/model/feature_extractor.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1d13034a9dbf1c2a"}},{"code_sha256_prefix":"3466e04858238f2e","entry":"convert","repo":"mcg-nju/vfimamba","repo_kind":"official","path":"Trainer_finetune.py","file_url":"https://github.com/mcg-nju/vfimamba/blob/HEAD/Trainer_finetune.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3466e04858238f2e"}},{"code_sha256_prefix":"23f41c2655d6295a","entry":"deconv","repo":"mcg-nju/vfimamba","repo_kind":"official","path":"model/refine.py","file_url":"https://github.com/mcg-nju/vfimamba/blob/HEAD/model/refine.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"23f41c2655d6295a"}},{"code_sha256_prefix":"1f6001e1c3b06556","entry":"downsample","repo":"mcg-nju/vfimamba","repo_kind":"official","path":"model/loss.py","file_url":"https://github.com/mcg-nju/vfimamba/blob/HEAD/model/loss.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1f6001e1c3b06556"}},{"code_sha256_prefix":"337a38a7be8077eb","entry":"gauss_kernel","repo":"mcg-nju/vfimamba","repo_kind":"official","path":"model/loss.py","file_url":"https://github.com/mcg-nju/vfimamba/blob/HEAD/model/loss.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"337a38a7be8077eb"}},{"code_sha256_prefix":"a21dfbe077bf6da8","entry":"init_model_config","repo":"mcg-nju/vfimamba","repo_kind":"official","path":"config.py","file_url":"https://github.com/mcg-nju/vfimamba/blob/HEAD/config.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a21dfbe077bf6da8"}},{"code_sha256_prefix":"23101166bb29f65e","entry":"upsample","repo":"mcg-nju/vfimamba","repo_kind":"official","path":"model/loss.py","file_url":"https://github.com/mcg-nju/vfimamba/blob/HEAD/model/loss.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"23101166bb29f65e"}},{"code_sha256_prefix":"662710159b487a9d","entry":"warp","repo":"mcg-nju/vfimamba","repo_kind":"official","path":"model/warplayer.py","file_url":"https://github.com/mcg-nju/vfimamba/blob/HEAD/model/warplayer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"662710159b487a9d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}