{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/belfusion-latent-diffusion-for-behavior","title":"BeLFusion: Latent Diffusion for Behavior-Driven Human Motion Prediction","arxiv_id":"2211.14304","date":"2022-11-25","proceeding":"ICCV 2023 1","authors":["German Barquero","Sergio Escalera","Cristina Palmero"],"abstract":"Stochastic human motion prediction (HMP) has generally been tackled with generative adversarial networks and variational autoencoders. Most prior works aim at predicting highly diverse movements in terms of the skeleton joints' dispersion. This has led to methods predicting fast and motion-divergent movements, which are often unrealistic and incoherent with past motion. Such methods also neglect contexts that need to anticipate diverse low-range behaviors, or actions, with subtle joint displacements. To address these issues, we present BeLFusion, a model that, for the first time, leverages latent diffusion models in HMP to sample from a latent space where behavior is disentangled from pose and motion. As a result, diversity is encouraged from a behavioral perspective. Thanks to our behavior coupler's ability to transfer sampled behavior to ongoing motion, BeLFusion's predictions display a variety of behaviors that are significantly more realistic than the state of the art. To support it, we introduce two metrics, the Area of the Cumulative Motion Distribution, and the Average Pairwise Distance Error, which are correlated to our definition of realism according to a qualitative study with 126 participants. Finally, we prove BeLFusion's generalization power in a new cross-dataset scenario for stochastic HMP.","url_abs":"https://arxiv.org/abs/2211.14304v3","url_pdf":"https://arxiv.org/pdf/2211.14304v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"belfusion-latent-diffusion-for-behavior","repo_url":"https://github.com/BarqueroGerman/BeLFusion","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"diversity","task_name":"Diversity"},{"task_slug":"human-pose-forecasting","task_name":"Human Pose Forecasting"},{"task_slug":"human-motion-prediction","task_name":"Human motion prediction"},{"task_slug":"motion-synthesis","task_name":"Motion Synthesis"},{"task_slug":"stochastic-human-motion-prediction","task_name":"Stochastic Human Motion Prediction"},{"task_slug":"motion-prediction","task_name":"motion prediction"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"latent-diffusion-model","method_name":"Latent Diffusion Model"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/human-pose-forecasting-on-amass","task":"Human Pose Forecasting","dataset":"AMASS","model":"BeLFusion","rank_in_archive_order":8,"of":11,"metrics":{"ADE":"0.513","APD":"9.376","APDE":"1.977","FDE":"0.560"},"uses_additional_data":false},{"leaderboard":"/sota/human-pose-forecasting-on-human36m","task":"Human Pose Forecasting","dataset":"Human3.6M","model":"BeLFusion","rank_in_archive_order":27,"of":33,"metrics":{"ADE":"372","APD":"7602","CMD":"5.988","FDE":"474","FID":"0.209","MMADE":"473","MMFDE":"507"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2211.14304","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}