{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/enhanced-bi-directional-motion-estimation-for","title":"Enhanced Bi-directional Motion Estimation for Video Frame Interpolation","arxiv_id":"2206.08572","date":"2022-06-17","proceeding":null,"authors":["Xin Jin","Longhai Wu","Guotao Shen","Youxin Chen","Jie Chen","Jayoon Koo","Cheul-hee Hahm"],"abstract":"We present a novel simple yet effective algorithm for motion-based video frame interpolation. Existing motion-based interpolation methods typically rely on a pre-trained optical flow model or a U-Net based pyramid network for motion estimation, which either suffer from large model size or limited capacity in handling complex and large motion cases. In this work, by carefully integrating intermediateoriented forward-warping, lightweight feature encoder, and correlation volume into a pyramid recurrent framework, we derive a compact model to simultaneously estimate the bidirectional motion between input frames. It is 15 times smaller in size than PWC-Net, yet enables more reliable and flexible handling of challenging motion cases. Based on estimated bi-directional motion, we forward-warp input frames and their context features to intermediate frame, and employ a synthesis network to estimate the intermediate frame from warped representations. Our method achieves excellent performance on a broad range of video frame interpolation benchmarks. Code and trained models are available at \\url{https://github.com/srcn-ivl/EBME}.","url_abs":"https://arxiv.org/abs/2206.08572v3","url_pdf":"https://arxiv.org/pdf/2206.08572v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"enhanced-bi-directional-motion-estimation-for","repo_url":"https://github.com/srcn-ivl/ebme","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"motion-estimation","task_name":"Motion Estimation"},{"task_slug":"optical-flow-estimation","task_name":"Optical Flow Estimation"},{"task_slug":"video-frame-interpolation","task_name":"Video Frame Interpolation"}],"methods":[{"method_slug":"concatenated-skip-connection","method_name":"Concatenated Skip Connection"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"u-net","method_name":"U-Net"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/video-frame-interpolation-on-msu-video-frame","task":"Video Frame Interpolation","dataset":"MSU Video Frame Interpolation","model":"EBME-H","rank_in_archive_order":7,"of":24,"metrics":{"LPIPS":"0.024","MS-SSIM":"0.958","PSNR":"28.77","SSIM":"0.931","VMAF":"68.20"},"uses_additional_data":false},{"leaderboard":"/sota/video-frame-interpolation-on-msu-video-frame","task":"Video Frame Interpolation","dataset":"MSU Video Frame Interpolation","model":"EBME","rank_in_archive_order":8,"of":24,"metrics":{"LPIPS":"0.028","MS-SSIM":"0.957","PSNR":"28.56","SSIM":"0.928","VMAF":"69.37"},"uses_additional_data":false},{"leaderboard":"/sota/video-frame-interpolation-on-snu-film-easy","task":"Video Frame Interpolation","dataset":"SNU-FILM (easy)","model":"EBME-H*","rank_in_archive_order":6,"of":8,"metrics":{"PSNR":"40.28","SSIM":"0.9910"},"uses_additional_data":false},{"leaderboard":"/sota/video-frame-interpolation-on-snu-film-extreme","task":"Video Frame Interpolation","dataset":"SNU-FILM (extreme)","model":"EBME-H*","rank_in_archive_order":8,"of":8,"metrics":{"PSNR":"25.40","SSIM":"0.863"},"uses_additional_data":false},{"leaderboard":"/sota/video-frame-interpolation-on-snu-film-hard","task":"Video Frame Interpolation","dataset":"SNU-FILM (hard)","model":"EBME-H*","rank_in_archive_order":8,"of":8,"metrics":{"PSNR":"30.64","SSIM":"0.937"},"uses_additional_data":false},{"leaderboard":"/sota/video-frame-interpolation-on-snu-film-medium","task":"Video Frame Interpolation","dataset":"SNU-FILM (medium)","model":"EBME-H*","rank_in_archive_order":7,"of":8,"metrics":{"PSNR":"36.07","SSIM":"0.980"},"uses_additional_data":false},{"leaderboard":"/sota/video-frame-interpolation-on-ucf101-1","task":"Video Frame Interpolation","dataset":"UCF101","model":"EBME-H*","rank_in_archive_order":7,"of":19,"metrics":{"PSNR":"35.41","SSIM":"0.970"},"uses_additional_data":false},{"leaderboard":"/sota/video-frame-interpolation-on-vimeo90k","task":"Video Frame Interpolation","dataset":"Vimeo90K","model":"EBME-H*","rank_in_archive_order":6,"of":23,"metrics":{"PSNR":"36.19","SSIM":"0.981"},"uses_additional_data":false},{"leaderboard":"/sota/video-frame-interpolation-on-x4k1000fps","task":"Video Frame Interpolation","dataset":"X4K1000FPS","model":"EBME-H*","rank_in_archive_order":13,"of":20,"metrics":{"PSNR":"29.46","SSIM":"0.902"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2206.08572","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}