{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/light-field-video-capture-using-a-learning","title":"Light Field Video Capture Using a Learning-Based Hybrid Imaging System","arxiv_id":"1705.02997","date":"2017-05-08","proceeding":null,"authors":["Ting-Chun Wang","Jun-Yan Zhu","Nima Khademi Kalantari","Alexei A. Efros","Ravi Ramamoorthi"],"abstract":"Light field cameras have many advantages over traditional cameras, as they\nallow the user to change various camera settings after capture. However,\ncapturing light fields requires a huge bandwidth to record the data: a modern\nlight field camera can only take three images per second. This prevents current\nconsumer light field cameras from capturing light field videos. Temporal\ninterpolation at such extreme scale (10x, from 3 fps to 30 fps) is infeasible\nas too much information will be entirely missing between adjacent frames.\nInstead, we develop a hybrid imaging system, adding another standard video\ncamera to capture the temporal information. Given a 3 fps light field sequence\nand a standard 30 fps 2D video, our system can then generate a full light field\nvideo at 30 fps. We adopt a learning-based approach, which can be decomposed\ninto two steps: spatio-temporal flow estimation and appearance estimation. The\nflow estimation propagates the angular information from the light field\nsequence to the 2D video, so we can warp input images to the target view. The\nappearance estimation then combines these warped images to output the final\npixels. The whole process is trained end-to-end using convolutional neural\nnetworks. Experimental results demonstrate that our algorithm outperforms\ncurrent video interpolation methods, enabling consumer light field videography,\nand making applications such as refocusing and parallax view generation\nachievable on videos for the first time.","url_abs":"http://arxiv.org/abs/1705.02997v1","url_pdf":"http://arxiv.org/pdf/1705.02997v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"light-field-video-capture-using-a-learning","repo_url":"https://github.com/junyanz/light-field-video","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1705.02997","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}