{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/disentangling-representations-using-gaussian","title":"Disentangling Multiple Features in Video Sequences using Gaussian Processes in Variational Autoencoders","arxiv_id":"2001.02408","date":"2020-01-08","proceeding":"ECCV 2020 8","authors":["Sarthak Bhagat","Shagun Uppal","Zhuyun Yin","Nengli Lim"],"abstract":"We introduce MGP-VAE (Multi-disentangled-features Gaussian Processes Variational AutoEncoder), a variational autoencoder which uses Gaussian processes (GP) to model the latent space for the unsupervised learning of disentangled representations in video sequences. We improve upon previous work by establishing a framework by which multiple features, static or dynamic, can be disentangled. Specifically we use fractional Brownian motions (fBM) and Brownian bridges (BB) to enforce an inter-frame correlation structure in each independent channel, and show that varying this structure enables one to capture different factors of variation in the data. We demonstrate the quality of our representations with experiments on three publicly available datasets, and also quantify the improvement using a video prediction task. Moreover, we introduce a novel geodesic loss function which takes into account the curvature of the data manifold to improve learning. Our experiments show that the combination of the improved representations with the novel loss function enable MGP-VAE to outperform the baselines in video prediction.","url_abs":"https://arxiv.org/abs/2001.02408v3","url_pdf":"https://arxiv.org/pdf/2001.02408v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"disentangling-representations-using-gaussian","repo_url":"https://github.com/SUTDBrainLab/MGP-VAE","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"gaussian-processes","task_name":"Gaussian Processes"},{"task_slug":"video-prediction","task_name":"Video Prediction"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/video-prediction-on-colored-dsprites","task":"Video Prediction","dataset":"Colored dSprites","model":"MGP-VAE (with geodesic loss)","rank_in_archive_order":1,"of":1,"metrics":{"MSE":"4.5"},"uses_additional_data":false},{"leaderboard":"/sota/video-prediction-on-sprites","task":"Video Prediction","dataset":"Sprites","model":"MGP-VAE (with geodesic loss)","rank_in_archive_order":1,"of":1,"metrics":{"MSE":"61.6"},"uses_additional_data":false}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}