{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/visual-dynamics-probabilistic-future-frame","title":"Visual Dynamics: Probabilistic Future Frame Synthesis via Cross Convolutional Networks","arxiv_id":"1607.02586","date":"2016-07-09","proceeding":"NeurIPS 2016 12","authors":["Tianfan Xue","Jiajun Wu","Katherine L. Bouman","William T. Freeman"],"abstract":"We study the problem of synthesizing a number of likely future frames from a\nsingle input image. In contrast to traditional methods, which have tackled this\nproblem in a deterministic or non-parametric way, we propose a novel approach\nthat models future frames in a probabilistic manner. Our probabilistic model\nmakes it possible for us to sample and synthesize many possible future frames\nfrom a single input image. Future frame synthesis is challenging, as it\ninvolves low- and high-level image and motion understanding. We propose a novel\nnetwork structure, namely a Cross Convolutional Network to aid in synthesizing\nfuture frames; this network structure encodes image and motion information as\nfeature maps and convolutional kernels, respectively. In experiments, our model\nperforms well on synthetic data, such as 2D shapes and animated game sprites,\nas well as on real-wold videos. We also show that our model can be applied to\ntasks such as visual analogy-making, and present an analysis of the learned\nnetwork representations.","url_abs":"http://arxiv.org/abs/1607.02586v1","url_pdf":"http://arxiv.org/pdf/1607.02586v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"visual-dynamics-probabilistic-future-frame","repo_url":"https://github.com/tensorflow/models","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"visual-dynamics-probabilistic-future-frame","repo_url":"https://github.com/tensorflow/models/tree/master/research/next_frame_prediction","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"visual-dynamics-probabilistic-future-frame","repo_url":"https://github.com/tfxue/visual-dynamics","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"torch","reach":{"status":"ok"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1607.02586","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}