{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-temporally-aware-interpolation-network-for","title":"A Temporally-Aware Interpolation Network for Video Frame Inpainting","arxiv_id":"1803.07218","date":"2018-03-20","proceeding":null,"authors":["Ximeng Sun","Ryan Szeto","Jason J. Corso"],"abstract":"We propose the first deep learning solution to video frame inpainting, a\nchallenging instance of the general video inpainting problem with applications\nin video editing, manipulation, and forensics. Our task is less ambiguous than\nframe interpolation and video prediction because we have access to both the\ntemporal context and a partial glimpse of the future, allowing us to better\nevaluate the quality of a model's predictions objectively. We devise a pipeline\ncomposed of two modules: a bidirectional video prediction module, and a\ntemporally-aware frame interpolation module. The prediction module makes two\nintermediate predictions of the missing frames, one conditioned on the\npreceding frames and the other conditioned on the following frames, using a\nshared convolutional LSTM-based encoder-decoder. The interpolation module\nblends the intermediate predictions to form the final result. Specifically, it\nutilizes time information and hidden activations from the video prediction\nmodule to resolve disagreements between the predictions. Our experiments\ndemonstrate that our approach produces more accurate and qualitatively\nsatisfying results than a state-of-the-art video prediction method and many\nstrong frame inpainting baselines.","url_abs":"http://arxiv.org/abs/1803.07218v2","url_pdf":"http://arxiv.org/pdf/1803.07218v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-temporally-aware-interpolation-network-for","repo_url":"https://github.com/sunxm2357/TAI_video_frame_inpainting","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"prediction","task_name":"Prediction"},{"task_slug":"video-editing","task_name":"Video Editing"},{"task_slug":"video-inpainting","task_name":"Video Inpainting"},{"task_slug":"video-prediction","task_name":"Video Prediction"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}