{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/folded-recurrent-neural-networks-for-future","title":"Folded Recurrent Neural Networks for Future Video Prediction","arxiv_id":"1712.00311","date":"2017-12-01","proceeding":"ECCV 2018 9","authors":["Marc Oliu","Javier Selva","Sergio Escalera"],"abstract":"Future video prediction is an ill-posed Computer Vision problem that recently\nreceived much attention. Its main challenges are the high variability in video\ncontent, the propagation of errors through time, and the non-specificity of the\nfuture frames: given a sequence of past frames there is a continuous\ndistribution of possible futures. This work introduces bijective Gated\nRecurrent Units, a double mapping between the input and output of a GRU layer.\nThis allows for recurrent auto-encoders with state sharing between encoder and\ndecoder, stratifying the sequence representation and helping to prevent\ncapacity problems. We show how with this topology only the encoder or decoder\nneeds to be applied for input encoding and prediction, respectively. This\nreduces the computational cost and avoids re-encoding the predictions when\ngenerating a sequence of frames, mitigating the propagation of errors.\nFurthermore, it is possible to remove layers from an already trained model,\ngiving an insight to the role performed by each layer and making the model more\nexplainable. We evaluate our approach on three video datasets, outperforming\nstate of the art prediction results on MMNIST and UCF101, and obtaining\ncompetitive results on KTH with 2 and 3 times less memory usage and\ncomputational cost than the best scored approach.","url_abs":"http://arxiv.org/abs/1712.00311v2","url_pdf":"http://arxiv.org/pdf/1712.00311v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"folded-recurrent-neural-networks-for-future","repo_url":"https://github.com/moliusimon/frnn","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"prediction","task_name":"Prediction"},{"task_slug":"specificity","task_name":"Specificity"},{"task_slug":"video-prediction","task_name":"Video Prediction"}],"methods":[{"method_slug":"gru","method_name":"GRU"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/video-prediction-on-human36m","task":"Video Prediction","dataset":"Human3.6M","model":"FRNN","rank_in_archive_order":9,"of":9,"metrics":{"MAE":"1901.1","MSE":"497.7","SSIM":"0.771"},"uses_additional_data":false},{"leaderboard":"/sota/video-prediction-on-kth","task":"Video Prediction","dataset":"KTH","model":"fRNN","rank_in_archive_order":28,"of":31,"metrics":{"Cond":"10","PSNR":"26.12","Pred":"20","SSIM":"0.771"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1712.00311","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}