{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/film-frame-interpolation-for-large-motion","title":"FILM: Frame Interpolation for Large Motion","arxiv_id":"2202.04901","date":"2022-02-10","proceeding":null,"authors":["Fitsum Reda","Janne Kontkanen","Eric Tabellion","Deqing Sun","Caroline Pantofaru","Brian Curless"],"abstract":"We present a frame interpolation algorithm that synthesizes multiple intermediate frames from two input images with large in-between motion. Recent methods use multiple networks to estimate optical flow or depth and a separate network dedicated to frame synthesis. This is often complex and requires scarce optical flow or depth ground-truth. In this work, we present a single unified network, distinguished by a multi-scale feature extractor that shares weights at all scales, and is trainable from frames alone. To synthesize crisp and pleasing frames, we propose to optimize our network with the Gram matrix loss that measures the correlation difference between feature maps. Our approach outperforms state-of-the-art methods on the Xiph large motion benchmark. We also achieve higher scores on Vimeo-90K, Middlebury and UCF101, when comparing to methods that use perceptual losses. We study the effect of weight sharing and of training with datasets of increasing motion range. Finally, we demonstrate our model's effectiveness in synthesizing high quality and temporally coherent videos on a challenging near-duplicate photos dataset. Codes and pre-trained models are available at https://film-net.github.io.","url_abs":"https://arxiv.org/abs/2202.04901v4","url_pdf":"https://arxiv.org/pdf/2202.04901v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"film-frame-interpolation-for-large-motion","repo_url":"https://github.com/google-research/frame-interpolation","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"film-frame-interpolation-for-large-motion","repo_url":"https://github.com/dajes/frame-interpolation-pytorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"optical-flow-estimation","task_name":"Optical Flow Estimation"},{"task_slug":"video-frame-interpolation","task_name":"Video Frame Interpolation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/video-frame-interpolation-on-msu-video-frame","task":"Video Frame Interpolation","dataset":"MSU Video Frame Interpolation","model":"FILM","rank_in_archive_order":10,"of":24,"metrics":{"LPIPS":"0.033","MS-SSIM":"0.948","PSNR":"28.11","SSIM":"0.928","VMAF":"68.68"},"uses_additional_data":false},{"leaderboard":"/sota/video-frame-interpolation-on-middlebury","task":"Video Frame Interpolation","dataset":"Middlebury","model":"FILM","rank_in_archive_order":10,"of":11,"metrics":{"PSNR":"37.52","SSIM":"0.966"},"uses_additional_data":false},{"leaderboard":"/sota/video-frame-interpolation-on-ucf101-1","task":"Video Frame Interpolation","dataset":"UCF101","model":"FILM","rank_in_archive_order":12,"of":19,"metrics":{"PSNR":"35.32","SSIM":"0.952"},"uses_additional_data":false},{"leaderboard":"/sota/video-frame-interpolation-on-vimeo90k","task":"Video Frame Interpolation","dataset":"Vimeo90K","model":"FILM","rank_in_archive_order":9,"of":23,"metrics":{"PSNR":"36.06","SSIM":"0.97"},"uses_additional_data":false},{"leaderboard":"/sota/video-frame-interpolation-on-xiph-2k","task":"Video Frame Interpolation","dataset":"Xiph-2K","model":"FILM","rank_in_archive_order":3,"of":4,"metrics":{"PSNR":"36.66","SSIM":"0.951"},"uses_additional_data":false},{"leaderboard":"/sota/video-frame-interpolation-on-xiph-4k-1","task":"Video Frame Interpolation","dataset":"Xiph-4k","model":"FILM","rank_in_archive_order":3,"of":3,"metrics":{"PSNR":"33.78","SSIM":"0.906"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2202.04901","atlas_url":"https://app.syntology.ai/?focus=2202.04901","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2202.04901"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dajes/frame-interpolation-pytorch","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/google-research/frame-interpolation","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":8,"ran_draft_wrong":3,"ran_honours":1,"unverified":8},"by_repo_kind":{"official":{"samples":6,"ran":2,"repositories":1},"listed":{"samples":14,"ran":10,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"33031ed5f24825b1","entry":"Conv2d","repo":"dajes/frame-interpolation-pytorch","repo_kind":"listed","path":"interpolator.py","file_url":"https://github.com/dajes/frame-interpolation-pytorch/blob/HEAD/interpolator.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"33031ed5f24825b1"}},{"code_sha256_prefix":"9ef67e49992f642f","entry":"FeatureExtractor","repo":"dajes/frame-interpolation-pytorch","repo_kind":"listed","path":"interpolator.py","file_url":"https://github.com/dajes/frame-interpolation-pytorch/blob/HEAD/interpolator.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9ef67e49992f642f"}},{"code_sha256_prefix":"a7349eca770ecdf9","entry":"FlowEstimator","repo":"dajes/frame-interpolation-pytorch","repo_kind":"listed","path":"interpolator.py","file_url":"https://github.com/dajes/frame-interpolation-pytorch/blob/HEAD/interpolator.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a7349eca770ecdf9"}},{"code_sha256_prefix":"5df4eeacc27fbe49","entry":"Fusion","repo":"dajes/frame-interpolation-pytorch","repo_kind":"listed","path":"interpolator.py","file_url":"https://github.com/dajes/frame-interpolation-pytorch/blob/HEAD/interpolator.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5df4eeacc27fbe49"}},{"code_sha256_prefix":"646c00a2c3ce3ec0","entry":"PyramidFlowEstimator","repo":"dajes/frame-interpolation-pytorch","repo_kind":"listed","path":"interpolator.py","file_url":"https://github.com/dajes/frame-interpolation-pytorch/blob/HEAD/interpolator.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"646c00a2c3ce3ec0"}},{"code_sha256_prefix":"c47da384c303149c","entry":"SubTreeExtractor","repo":"dajes/frame-interpolation-pytorch","repo_kind":"listed","path":"interpolator.py","file_url":"https://github.com/dajes/frame-interpolation-pytorch/blob/HEAD/interpolator.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c47da384c303149c"}},{"code_sha256_prefix":"5e996c1e85b97cc1","entry":"build_image_pyramid","repo":"dajes/frame-interpolation-pytorch","repo_kind":"listed","path":"interpolator.py","file_url":"https://github.com/dajes/frame-interpolation-pytorch/blob/HEAD/interpolator.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5e996c1e85b97cc1"}},{"code_sha256_prefix":"e5a70acb4070eb0f","entry":"concatenate_pyramids","repo":"dajes/frame-interpolation-pytorch","repo_kind":"listed","path":"interpolator.py","file_url":"https://github.com/dajes/frame-interpolation-pytorch/blob/HEAD/interpolator.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e5a70acb4070eb0f"}},{"code_sha256_prefix":"6ea74e810cda7a86","entry":"flow_pyramid_synthesis","repo":"dajes/frame-interpolation-pytorch","repo_kind":"listed","path":"interpolator.py","file_url":"https://github.com/dajes/frame-interpolation-pytorch/blob/HEAD/interpolator.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6ea74e810cda7a86"}},{"code_sha256_prefix":"f8ee6422386d84b6","entry":"get_channels_at_level","repo":"dajes/frame-interpolation-pytorch","repo_kind":"listed","path":"interpolator.py","file_url":"https://github.com/dajes/frame-interpolation-pytorch/blob/HEAD/interpolator.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f8ee6422386d84b6"}},{"code_sha256_prefix":"36bbb37e74c4a4dc","entry":"l1_loss","repo":"google-research/frame-interpolation","repo_kind":"official","path":"losses/losses.py","file_url":"https://github.com/google-research/frame-interpolation/blob/HEAD/losses/losses.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"36bbb37e74c4a4dc"}},{"code_sha256_prefix":"41eb7c711cf87551","entry":"patches_to_image","repo":"google-research/frame-interpolation","repo_kind":"official","path":"eval/interpolator.py","file_url":"https://github.com/google-research/frame-interpolation/blob/HEAD/eval/interpolator.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"41eb7c711cf87551"}},{"code_sha256_prefix":"e48dec25e2c77c29","entry":"Interpolator","repo":"dajes/frame-interpolation-pytorch","repo_kind":"listed","path":"interpolator.py","file_url":"https://github.com/dajes/frame-interpolation-pytorch/blob/HEAD/interpolator.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e48dec25e2c77c29"}},{"code_sha256_prefix":"62b36be97a57cd56","entry":"generate_image_triplet_example","repo":"google-research/frame-interpolation","repo_kind":"official","path":"datasets/util.py","file_url":"https://github.com/google-research/frame-interpolation/blob/HEAD/datasets/util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"62b36be97a57cd56"}},{"code_sha256_prefix":"81e6e74325f0e8ca","entry":"image_to_patches","repo":"google-research/frame-interpolation","repo_kind":"official","path":"eval/interpolator.py","file_url":"https://github.com/google-research/frame-interpolation/blob/HEAD/eval/interpolator.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"81e6e74325f0e8ca"}},{"code_sha256_prefix":"12d32fa9bc246c8f","entry":"multiply_pyramid","repo":"dajes/frame-interpolation-pytorch","repo_kind":"listed","path":"interpolator.py","file_url":"https://github.com/dajes/frame-interpolation-pytorch/blob/HEAD/interpolator.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"12d32fa9bc246c8f"}},{"code_sha256_prefix":"6c403027f71d5c0e","entry":"pyramid_warp","repo":"dajes/frame-interpolation-pytorch","repo_kind":"listed","path":"interpolator.py","file_url":"https://github.com/dajes/frame-interpolation-pytorch/blob/HEAD/interpolator.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6c403027f71d5c0e"}},{"code_sha256_prefix":"c4b4c0fc22dc8ce1","entry":"style_loss","repo":"google-research/frame-interpolation","repo_kind":"official","path":"losses/vgg19_loss.py","file_url":"https://github.com/google-research/frame-interpolation/blob/HEAD/losses/vgg19_loss.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c4b4c0fc22dc8ce1"}},{"code_sha256_prefix":"56ee3540e67f1685","entry":"vgg_loss","repo":"google-research/frame-interpolation","repo_kind":"official","path":"losses/vgg19_loss.py","file_url":"https://github.com/google-research/frame-interpolation/blob/HEAD/losses/vgg19_loss.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"56ee3540e67f1685"}},{"code_sha256_prefix":"b6d4f892f64d3fe9","entry":"warp","repo":"dajes/frame-interpolation-pytorch","repo_kind":"listed","path":"interpolator.py","file_url":"https://github.com/dajes/frame-interpolation-pytorch/blob/HEAD/interpolator.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b6d4f892f64d3fe9"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}