{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mambatalk-efficient-holistic-gesture","title":"MambaTalk: Efficient Holistic Gesture Synthesis with Selective State Space Models","arxiv_id":"2403.09471","date":"2024-03-14","proceeding":null,"authors":["Zunnan Xu","Yukang Lin","Haonan Han","Sicheng Yang","Ronghui Li","Yachao Zhang","Xiu Li"],"abstract":"Gesture synthesis is a vital realm of human-computer interaction, with wide-ranging applications across various fields like film, robotics, and virtual reality. Recent advancements have utilized the diffusion model and attention mechanisms to improve gesture synthesis. However, due to the high computational complexity of these techniques, generating long and diverse sequences with low latency remains a challenge. We explore the potential of state space models (SSMs) to address the challenge, implementing a two-stage modeling strategy with discrete motion priors to enhance the quality of gestures. Leveraging the foundational Mamba block, we introduce MambaTalk, enhancing gesture diversity and rhythm through multimodal integration. Extensive experiments demonstrate that our method matches or exceeds the performance of state-of-the-art models. Our project is publicly available at https://kkakkkka.github.io/MambaTalk","url_abs":"https://arxiv.org/abs/2403.09471v6","url_pdf":"https://arxiv.org/pdf/2403.09471v6.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mambatalk-efficient-holistic-gesture","repo_url":"https://github.com/kkakkkka/MambaTalk","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"3d-face-animation","task_name":"3D Face Animation"},{"task_slug":"diversity","task_name":"Diversity"},{"task_slug":"gesture-generation","task_name":"Gesture Generation"},{"task_slug":"mamba","task_name":"Mamba"},{"task_slug":"rhythm","task_name":"Rhythm"},{"task_slug":"state-space-models","task_name":"State Space Models"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/3d-face-animation-on-beat2","task":"3D Face Animation","dataset":"BEAT2","model":"MambaTalk","rank_in_archive_order":1,"of":5,"metrics":{"MSE":"6.289"},"uses_additional_data":false},{"leaderboard":"/sota/gesture-generation-on-beat2","task":"Gesture Generation","dataset":"BEAT2","model":"MambaTalk","rank_in_archive_order":7,"of":14,"metrics":{"FGD":"0.5366"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2403.09471","atlas_url":"https://app.syntology.ai/?focus=2403.09471","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.09471"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kkakkkka/MambaTalk","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":1},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"4f043bf1bec12652","entry":"get_loss_func","repo":"kkakkkka/MambaTalk","repo_kind":"official","path":"optimizers/loss_factory.py","file_url":"https://github.com/kkakkkka/MambaTalk/blob/HEAD/optimizers/loss_factory.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4f043bf1bec12652"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}