{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/motiongpt-human-motion-as-a-foreign-language","title":"MotionGPT: Human Motion as a Foreign Language","arxiv_id":"2306.14795","date":"2023-06-26","proceeding":"NeurIPS 2023 11","authors":["Biao Jiang","Xin Chen","Wen Liu","Jingyi Yu","Gang Yu","Tao Chen"],"abstract":"Though the advancement of pre-trained large language models unfolds, the exploration of building a unified model for language and other multi-modal data, such as motion, remains challenging and untouched so far. Fortunately, human motion displays a semantic coupling akin to human language, often perceived as a form of body language. By fusing language data with large-scale motion models, motion-language pre-training that can enhance the performance of motion-related tasks becomes feasible. Driven by this insight, we propose MotionGPT, a unified, versatile, and user-friendly motion-language model to handle multiple motion-relevant tasks. Specifically, we employ the discrete vector quantization for human motion and transfer 3D motion into motion tokens, similar to the generation process of word tokens. Building upon this \"motion vocabulary\", we perform language modeling on both motion and text in a unified manner, treating human motion as a specific language. Moreover, inspired by prompt learning, we pre-train MotionGPT with a mixture of motion-language data and fine-tune it on prompt-based question-and-answer tasks. Extensive experiments demonstrate that MotionGPT achieves state-of-the-art performances on multiple motion tasks including text-driven motion generation, motion captioning, motion prediction, and motion in-between.","url_abs":"https://arxiv.org/abs/2306.14795v2","url_pdf":"https://arxiv.org/pdf/2306.14795v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"motiongpt-human-motion-as-a-foreign-language","repo_url":"https://github.com/openmotionlab/motiongpt","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"motiongpt-human-motion-as-a-foreign-language","repo_url":"https://github.com/open3da/ll3da","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"motiongpt-human-motion-as-a-foreign-language","repo_url":"https://github.com/openmeshlab/meshxl","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"motion-captioning","task_name":"Motion Captioning"},{"task_slug":"motion-generation","task_name":"Motion Generation"},{"task_slug":"prompt-learning","task_name":"Prompt Learning"},{"task_slug":"quantization","task_name":"Quantization"},{"task_slug":"motion-prediction","task_name":"motion prediction"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/motion-captioning-on-humanml3d","task":"Motion Captioning","dataset":"HumanML3D","model":"MotionGPT","rank_in_archive_order":4,"of":4,"metrics":{"BERTScore":"32.4","BLEU-4":"12.47"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2306.14795","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2306.14795"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/openmotionlab/motiongpt","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/openmeshlab/meshxl","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/open3da/ll3da","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":1,"unverified":8},"by_repo_kind":{"official":{"samples":9,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"e80c033b31c865ac","entry":"parse_args","repo":"openmotionlab/motiongpt","repo_kind":"official","path":"render.py","file_url":"https://github.com/openmotionlab/motiongpt/blob/HEAD/render.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e80c033b31c865ac"}},{"code_sha256_prefix":"6b6950756dabbc50","entry":"build_callbacks","repo":"openmotionlab/motiongpt","repo_kind":"official","path":"mGPT/callback.py","file_url":"https://github.com/openmotionlab/motiongpt/blob/HEAD/mGPT/callback.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6b6950756dabbc50"}},{"code_sha256_prefix":"e241774940f0b133","entry":"getCheckpointCallback","repo":"openmotionlab/motiongpt","repo_kind":"official","path":"mGPT/callback.py","file_url":"https://github.com/openmotionlab/motiongpt/blob/HEAD/mGPT/callback.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e241774940f0b133"}},{"code_sha256_prefix":"1785dd97dc89b367","entry":"get_module_config","repo":"openmotionlab/motiongpt","repo_kind":"official","path":"mGPT/config.py","file_url":"https://github.com/openmotionlab/motiongpt/blob/HEAD/mGPT/config.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1785dd97dc89b367"}},{"code_sha256_prefix":"9877bd3cecd6081f","entry":"get_obj_from_str","repo":"openmotionlab/motiongpt","repo_kind":"official","path":"mGPT/config.py","file_url":"https://github.com/openmotionlab/motiongpt/blob/HEAD/mGPT/config.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9877bd3cecd6081f"}},{"code_sha256_prefix":"4c5959826374192f","entry":"instantiate_from_config","repo":"openmotionlab/motiongpt","repo_kind":"official","path":"mGPT/config.py","file_url":"https://github.com/openmotionlab/motiongpt/blob/HEAD/mGPT/config.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4c5959826374192f"}},{"code_sha256_prefix":"caf019444a46328d","entry":"load_and_remove_chumpy_dep","repo":"openmotionlab/motiongpt","repo_kind":"official","path":"prepare/merge_smplh_mano.py","file_url":"https://github.com/openmotionlab/motiongpt/blob/HEAD/prepare/merge_smplh_mano.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"caf019444a46328d"}},{"code_sha256_prefix":"6981c5dbb9382f84","entry":"load_npz_into_dict","repo":"openmotionlab/motiongpt","repo_kind":"official","path":"prepare/merge_smplh_mano.py","file_url":"https://github.com/openmotionlab/motiongpt/blob/HEAD/prepare/merge_smplh_mano.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6981c5dbb9382f84"}},{"code_sha256_prefix":"09694aa91bbe4c81","entry":"remove_chumpy_dep","repo":"openmotionlab/motiongpt","repo_kind":"official","path":"prepare/merge_smplh_mano.py","file_url":"https://github.com/openmotionlab/motiongpt/blob/HEAD/prepare/merge_smplh_mano.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"09694aa91bbe4c81"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}