{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/exploring-text-to-motion-generation-with","title":"Exploring Text-to-Motion Generation with Human Preference","arxiv_id":"2404.09445","date":"2024-04-15","proceeding":null,"authors":["Jenny Sheng","Matthieu Lin","Andrew Zhao","Kevin Pruvost","Yu-Hui Wen","Yangguang Li","Gao Huang","Yong-Jin Liu"],"abstract":"This paper presents an exploration of preference learning in text-to-motion generation. We find that current improvements in text-to-motion generation still rely on datasets requiring expert labelers with motion capture systems. Instead, learning from human preference data does not require motion capture systems; a labeler with no expertise simply compares two generated motions. This is particularly efficient because evaluating the model's output is easier than gathering the motion that performs a desired task (e.g. backflip). To pioneer the exploration of this paradigm, we annotate 3,528 preference pairs generated by MotionGPT, marking the first effort to investigate various algorithms for learning from preference data. In particular, our exploration highlights important design choices when using preference data. Additionally, our experimental results show that preference learning has the potential to greatly improve current text-to-motion generative models. Our code and dataset are publicly available at https://github.com/THU-LYJ-Lab/InstructMotion}{https://github.com/THU-LYJ-Lab/InstructMotion to further facilitate research in this area.","url_abs":"https://arxiv.org/abs/2404.09445v1","url_pdf":"https://arxiv.org/pdf/2404.09445v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"exploring-text-to-motion-generation-with","repo_url":"https://github.com/thu-lyj-lab/instructmotion","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"motion-generation","task_name":"Motion Generation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2404.09445","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2404.09445"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/thu-lyj-lab/instructmotion","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":4,"unverified":2},"by_repo_kind":{"official":{"samples":6,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"c0eb0f4a912ff31f","entry":"load_example_input","repo":"thu-lyj-lab/instructmotion","repo_kind":"official","path":"MotionGPT/generate_npy.py","file_url":"https://github.com/thu-lyj-lab/instructmotion/blob/HEAD/MotionGPT/generate_npy.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c0eb0f4a912ff31f"}},{"code_sha256_prefix":"0797e250e681c777","entry":"load_pretrained_reward_rm","repo":"thu-lyj-lab/instructmotion","repo_kind":"official","path":"src/utils/motiongpt_utils.py","file_url":"https://github.com/thu-lyj-lab/instructmotion/blob/HEAD/src/utils/motiongpt_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0797e250e681c777"}},{"code_sha256_prefix":"65af45c09615b0ba","entry":"motion_token_to_string","repo":"thu-lyj-lab/instructmotion","repo_kind":"official","path":"MotionGPT/generate_npy.py","file_url":"https://github.com/thu-lyj-lab/instructmotion/blob/HEAD/MotionGPT/generate_npy.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"65af45c09615b0ba"}},{"code_sha256_prefix":"e80c033b31c865ac","entry":"parse_args","repo":"thu-lyj-lab/instructmotion","repo_kind":"official","path":"MotionGPT/render.py","file_url":"https://github.com/thu-lyj-lab/instructmotion/blob/HEAD/MotionGPT/render.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e80c033b31c865ac"}},{"code_sha256_prefix":"59126a0d078805a0","entry":"get_motion_tokenizer","repo":"thu-lyj-lab/instructmotion","repo_kind":"official","path":"src/utils/motiongpt_utils.py","file_url":"https://github.com/thu-lyj-lab/instructmotion/blob/HEAD/src/utils/motiongpt_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"59126a0d078805a0"}},{"code_sha256_prefix":"77ac2997bd266444","entry":"load_pretrained_reward_ppo","repo":"thu-lyj-lab/instructmotion","repo_kind":"official","path":"src/utils/motiongpt_utils.py","file_url":"https://github.com/thu-lyj-lab/instructmotion/blob/HEAD/src/utils/motiongpt_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"77ac2997bd266444"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}