{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/softmax-splatting-for-video-frame","title":"Softmax Splatting for Video Frame Interpolation","arxiv_id":"2003.05534","date":"2020-03-11","proceeding":"CVPR 2020 6","authors":["Simon Niklaus","Feng Liu"],"abstract":"Differentiable image sampling in the form of backward warping has seen broad adoption in tasks like depth estimation and optical flow prediction. In contrast, how to perform forward warping has seen less attention, partly due to additional challenges such as resolving the conflict of mapping multiple pixels to the same target location in a differentiable way. We propose softmax splatting to address this paradigm shift and show its effectiveness on the application of frame interpolation. Specifically, given two input frames, we forward-warp the frames and their feature pyramid representations based on an optical flow estimate using softmax splatting. In doing so, the softmax splatting seamlessly handles cases where multiple source pixels map to the same target location. We then use a synthesis network to predict the interpolation result from the warped representations. Our softmax splatting allows us to not only interpolate frames at an arbitrary time but also to fine tune the feature pyramid and the optical flow. We show that our synthesis approach, empowered by softmax splatting, achieves new state-of-the-art results for video frame interpolation.","url_abs":"https://arxiv.org/abs/2003.05534v1","url_pdf":"https://arxiv.org/pdf/2003.05534v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"softmax-splatting-for-video-frame","repo_url":"https://github.com/sniklaus/softmax-splatting","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"softmax-splatting-for-video-frame","repo_url":"https://github.com/hperrot/splatting","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"depth-estimation","task_name":"Depth Estimation"},{"task_slug":"optical-flow-estimation","task_name":"Optical Flow Estimation"},{"task_slug":"video-frame-interpolation","task_name":"Video Frame Interpolation"}],"methods":[{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/video-frame-interpolation-on-middlebury","task":"Video Frame Interpolation","dataset":"Middlebury","model":"SoftSplat","rank_in_archive_order":2,"of":11,"metrics":{"Interpolation Error":"4.223","PSNR":"38.42","SSIM":"0.971"},"uses_additional_data":false},{"leaderboard":"/sota/video-frame-interpolation-on-ucf101-1","task":"Video Frame Interpolation","dataset":"UCF101","model":"SoftSplat","rank_in_archive_order":8,"of":19,"metrics":{"PSNR":"35.39","SSIM":"0.952"},"uses_additional_data":false},{"leaderboard":"/sota/video-frame-interpolation-on-vimeo90k","task":"Video Frame Interpolation","dataset":"Vimeo90K","model":"SoftSplat","rank_in_archive_order":8,"of":23,"metrics":{"PSNR":"36.10","SSIM":"0.970"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2003.05534","atlas_url":"https://app.syntology.ai/?focus=2003.05534","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2003.05534"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sniklaus/softmax-splatting","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/hperrot/splatting","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":1},"by_repo_kind":{"listed":{"samples":1,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"37ca92ada872613a","entry":"splatting_function","repo":"hperrot/splatting","repo_kind":"listed","path":"splatting/splatting.py","file_url":"https://github.com/hperrot/splatting/blob/HEAD/splatting/splatting.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"37ca92ada872613a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}