{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/remodiffuse-retrieval-augmented-motion","title":"ReMoDiffuse: Retrieval-Augmented Motion Diffusion Model","arxiv_id":"2304.01116","date":"2023-04-03","proceeding":"ICCV 2023 1","authors":["Mingyuan Zhang","Xinying Guo","Liang Pan","Zhongang Cai","Fangzhou Hong","Huirong Li","Lei Yang","Ziwei Liu"],"abstract":"3D human motion generation is crucial for creative industry. Recent advances rely on generative models with domain knowledge for text-driven motion generation, leading to substantial progress in capturing common motions. However, the performance on more diverse motions remains unsatisfactory. In this work, we propose ReMoDiffuse, a diffusion-model-based motion generation framework that integrates a retrieval mechanism to refine the denoising process. ReMoDiffuse enhances the generalizability and diversity of text-driven motion generation with three key designs: 1) Hybrid Retrieval finds appropriate references from the database in terms of both semantic and kinematic similarities. 2) Semantic-Modulated Transformer selectively absorbs retrieval knowledge, adapting to the difference between retrieved samples and the target motion sequence. 3) Condition Mixture better utilizes the retrieval database during inference, overcoming the scale sensitivity in classifier-free guidance. Extensive experiments demonstrate that ReMoDiffuse outperforms state-of-the-art methods by balancing both text-motion consistency and motion quality, especially for more diverse motion generation.","url_abs":"https://arxiv.org/abs/2304.01116v1","url_pdf":"https://arxiv.org/pdf/2304.01116v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"remodiffuse-retrieval-augmented-motion","repo_url":"https://github.com/mingyuan-zhang/ReMoDiffuse","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"denoising","task_name":"Denoising"},{"task_slug":"diversity","task_name":"Diversity"},{"task_slug":"motion-generation","task_name":"Motion Generation"},{"task_slug":"motion-synthesis","task_name":"Motion Synthesis"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"model","task_name":"model"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/motion-synthesis-on-humanml3d","task":"Motion Synthesis","dataset":"HumanML3D","model":"ReMoDiffuse","rank_in_archive_order":15,"of":37,"metrics":{"Diversity":"9.018","FID":"0.103","Multimodality":"1.795","R Precision Top3":"0.795"},"uses_additional_data":false},{"leaderboard":"/sota/motion-synthesis-on-kit-motion-language","task":"Motion Synthesis","dataset":"KIT Motion-Language","model":"ReMoDiffuse","rank_in_archive_order":3,"of":31,"metrics":{"Diversity":"10.80","FID":"0.155","Multimodality":"1.239","R Precision Top3":"0.765"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2304.01116","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}