{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/timelens-event-based-video-frame","title":"TimeLens: Event-based Video Frame Interpolation","arxiv_id":"2106.07286","date":"2021-06-14","proceeding":null,"authors":["Stepan Tulyakov","Daniel Gehrig","Stamatios Georgoulis","Julius Erbach","Mathias Gehrig","Yuanyou Li","Davide Scaramuzza"],"abstract":"State-of-the-art frame interpolation methods generate intermediate frames by inferring object motions in the image from consecutive key-frames. In the absence of additional information, first-order approximations, i.e. optical flow, must be used, but this choice restricts the types of motions that can be modeled, leading to errors in highly dynamic scenarios. Event cameras are novel sensors that address this limitation by providing auxiliary visual information in the blind-time between frames. They asynchronously measure per-pixel brightness changes and do this with high temporal resolution and low latency. Event-based frame interpolation methods typically adopt a synthesis-based approach, where predicted frame residuals are directly applied to the key-frames. However, while these approaches can capture non-linear motions they suffer from ghosting and perform poorly in low-texture regions with few events. Thus, synthesis-based and flow-based approaches are complementary. In this work, we introduce Time Lens, a novel indicates equal contribution method that leverages the advantages of both. We extensively evaluate our method on three synthetic and two real benchmarks where we show an up to 5.21 dB improvement in terms of PSNR over state-of-the-art frame-based and event-based methods. Finally, we release a new large-scale dataset in highly dynamic scenarios, aimed at pushing the limits of existing methods.","url_abs":"https://arxiv.org/abs/2106.07286v1","url_pdf":"https://arxiv.org/pdf/2106.07286v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"timelens-event-based-video-frame","repo_url":"https://github.com/uzh-rpg/rpg_timelens","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"optical-flow-estimation","task_name":"Optical Flow Estimation"},{"task_slug":"video-frame-interpolation","task_name":"Video Frame Interpolation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2106.07286","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2106.07286"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/uzh-rpg/rpg_timelens","reach":null}],"summary":{"ran_honours":1,"ran":2,"unverified":2},"by_repo_kind":{"official":{"samples":5,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"1c047a66c72dcb57","entry":"_pack","repo":"uzh-rpg/rpg_timelens","repo_kind":"official","path":"timelens/fusion_network.py","file_url":"https://github.com/uzh-rpg/rpg_timelens/blob/HEAD/timelens/fusion_network.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1c047a66c72dcb57"}},{"code_sha256_prefix":"f4a6e59f5b7ab6cb","entry":"down","repo":"uzh-rpg/rpg_timelens","repo_kind":"official","path":"timelens/fusion_network.py","file_url":"https://github.com/uzh-rpg/rpg_timelens/blob/HEAD/timelens/fusion_network.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f4a6e59f5b7ab6cb"}},{"code_sha256_prefix":"bddca0ce050f42ba","entry":"up","repo":"uzh-rpg/rpg_timelens","repo_kind":"official","path":"timelens/fusion_network.py","file_url":"https://github.com/uzh-rpg/rpg_timelens/blob/HEAD/timelens/fusion_network.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bddca0ce050f42ba"}},{"code_sha256_prefix":"ad0637581bdcf818","entry":"Fusion","repo":"uzh-rpg/rpg_timelens","repo_kind":"official","path":"timelens/fusion_network.py","file_url":"https://github.com/uzh-rpg/rpg_timelens/blob/HEAD/timelens/fusion_network.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ad0637581bdcf818"}},{"code_sha256_prefix":"362d4f55493237e8","entry":"UNet","repo":"uzh-rpg/rpg_timelens","repo_kind":"official","path":"timelens/fusion_network.py","file_url":"https://github.com/uzh-rpg/rpg_timelens/blob/HEAD/timelens/fusion_network.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"362d4f55493237e8"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}