{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/deep-predictive-coding-networks-for-video","title":"Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning","arxiv_id":"1605.08104","date":"2016-05-25","proceeding":null,"authors":["William Lotter","Gabriel Kreiman","David Cox"],"abstract":"While great strides have been made in using deep learning algorithms to solve\nsupervised learning tasks, the problem of unsupervised learning - leveraging\nunlabeled examples to learn about the structure of a domain - remains a\ndifficult unsolved challenge. Here, we explore prediction of future frames in a\nvideo sequence as an unsupervised learning rule for learning about the\nstructure of the visual world. We describe a predictive neural network\n(\"PredNet\") architecture that is inspired by the concept of \"predictive coding\"\nfrom the neuroscience literature. These networks learn to predict future frames\nin a video sequence, with each layer in the network making local predictions\nand only forwarding deviations from those predictions to subsequent network\nlayers. We show that these networks are able to robustly learn to predict the\nmovement of synthetic (rendered) objects, and that in doing so, the networks\nlearn internal representations that are useful for decoding latent object\nparameters (e.g. pose) that support object recognition with fewer training\nviews. We also show that these networks can scale to complex natural image\nstreams (car-mounted camera videos), capturing key aspects of both egocentric\nmovement and the movement of objects in the visual scene, and the\nrepresentation learned in this setting is useful for estimating the steering\nangle. Altogether, these results suggest that prediction represents a powerful\nframework for unsupervised learning, allowing for implicit learning of object\nand scene structure.","url_abs":"http://arxiv.org/abs/1605.08104v5","url_pdf":"http://arxiv.org/pdf/1605.08104v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"deep-predictive-coding-networks-for-video","repo_url":"https://github.com/Crazy-Jack/PredNet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"deep-predictive-coding-networks-for-video","repo_url":"https://github.com/IcarusWizard/prednet-paddle","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"paddle","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"deep-predictive-coding-networks-for-video","repo_url":"https://github.com/LanaSina/chainer_prednet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"deep-predictive-coding-networks-for-video","repo_url":"https://github.com/Mikkil5112/PredNet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"deep-predictive-coding-networks-for-video","repo_url":"https://github.com/ZdenekStraka/precnet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"deep-predictive-coding-networks-for-video","repo_url":"https://github.com/coxlab/prednet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"deep-predictive-coding-networks-for-video","repo_url":"https://github.com/jhon-zelada/proyect","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"deep-predictive-coding-networks-for-video","repo_url":"https://github.com/kn-lambda/Pred-Net","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"deep-predictive-coding-networks-for-video","repo_url":"https://github.com/kunimasa-kawasaki/chainer-prednet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"deep-predictive-coding-networks-for-video","repo_url":"https://github.com/kunimasa-kawasaki/keras-prednet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"deep-predictive-coding-networks-for-video","repo_url":"https://github.com/marisancans/frame-predict","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"deep-predictive-coding-networks-for-video","repo_url":"https://github.com/microsoft/MLOps_VideoAnomalyDetection","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"deep-predictive-coding-networks-for-video","repo_url":"https://github.com/nikolasmcneal/music-prediction","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"deep-predictive-coding-networks-for-video","repo_url":"https://github.com/pallekc91/InceptionInspiredLSTMforPredNet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"deep-predictive-coding-networks-for-video","repo_url":"https://github.com/ramyamounir/PredNet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"deep-predictive-coding-networks-for-video","repo_url":"https://github.com/thefonseca/predictive-coding","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"unanswered"}},{"paper_slug":"deep-predictive-coding-networks-for-video","repo_url":"https://github.com/zahra-bk/MLOps_VideoAnomalyDetection","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"object-recognition","task_name":"Object Recognition"},{"task_slug":"video-prediction","task_name":"Video Prediction"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1605.08104","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1605.08104"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/nikolasmcneal/music-prediction","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/LanaSina/chainer_prednet","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kn-lambda/Pred-Net","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/coxlab/prednet","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/microsoft/MLOps_VideoAnomalyDetection","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/thefonseca/predictive-coding","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kunimasa-kawasaki/keras-prednet","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ZdenekStraka/precnet","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kunimasa-kawasaki/chainer-prednet","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jhon-zelada/proyect","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/IcarusWizard/prednet-paddle","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Crazy-Jack/PredNet","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/marisancans/frame-predict","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/pallekc91/InceptionInspiredLSTMforPredNet","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zahra-bk/MLOps_VideoAnomalyDetection","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ramyamounir/PredNet","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Mikkil5112/PredNet","reach":{"status":"ok"}}],"summary":{"unverified":6},"by_repo_kind":{"listed":{"samples":6,"ran":0,"repositories":4}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"2d25e8a5c964ab05","entry":"extrap_loss","repo":"coxlab/prednet","repo_kind":"listed","path":"kitti_extrap_finetune.py","file_url":"https://github.com/coxlab/prednet/blob/HEAD/kitti_extrap_finetune.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2d25e8a5c964ab05"}},{"code_sha256_prefix":"1aece28aec3ebd30","entry":"load_gif","repo":"IcarusWizard/prednet-paddle","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/IcarusWizard/prednet-paddle/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1aece28aec3ebd30"}},{"code_sha256_prefix":"a6ccdb74a8f5c40d","entry":"process_im","repo":"jhon-zelada/proyect","repo_kind":"listed","path":"data_preparation.py","file_url":"https://github.com/jhon-zelada/proyect/blob/HEAD/data_preparation.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"a6ccdb74a8f5c40d"}},{"code_sha256_prefix":"56120e6b0dd1dcd4","entry":"process_im","repo":"microsoft/MLOps_VideoAnomalyDetection","repo_kind":"listed","path":"data_preparation.py","file_url":"https://github.com/microsoft/MLOps_VideoAnomalyDetection/blob/HEAD/data_preparation.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"56120e6b0dd1dcd4"}},{"code_sha256_prefix":"b1266bac1a8f0c3a","entry":"run","repo":"jhon-zelada/proyect","repo_kind":"listed","path":"deployment/score.py","file_url":"https://github.com/jhon-zelada/proyect/blob/HEAD/deployment/score.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"b1266bac1a8f0c3a"}},{"code_sha256_prefix":"fed7e777b099718c","entry":"run","repo":"microsoft/MLOps_VideoAnomalyDetection","repo_kind":"listed","path":"deployment/score.py","file_url":"https://github.com/microsoft/MLOps_VideoAnomalyDetection/blob/HEAD/deployment/score.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fed7e777b099718c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}