Browse State-of-the-Art › Gesture Generation
Gesture Generation
46 papers with code · 4 benchmarks · 7 datasets archive 2025-07-28
Generation of gestures, as a sequence of 3d poses
Description from the archive archive 2025-07-28.
Benchmarks archive 2025-07-28
4 leaderboard tables shown for this task, 4 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.
| Dataset | Best model (first row in archive order) | Paper | Code | Syntology | Compare |
|---|---|---|---|---|---|
| BEAT2 (14 rows) | Intentional Gesture | Intentional Gesture: Deliver Your Intentions with Gestures for Speech | code | — | Compare |
| TED Gesture Dataset (6 rows) | AQ-GT | AQ-GT: a Temporally Aligned and Quantized GRU-Transformer for... | code | — | Compare |
| BEAT (5 rows) | CaMN | BEAT: A Large-Scale Semantic and Emotional Multi-Modal Dataset for... | code | Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified | Compare |
| DVS128 Gesture (1 row) | ECSNet | Ecsnet: Spatio-temporal feature learning for event camera | code | — | Compare |
Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. Each line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the row's paper, the line starts with that repository's state (“official” is the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. A dash means no Syntology record for that paper, not a recorded non-run. Each “ran” line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
7 datasets whose archive record lists this task, ordered by the archive's paper count.
Subtasks archive 2025-07-28
No subtask under this task in the archive's task tree.
Parent tasks archive 2025-07-28
Papers archive 2025-07-28
archive papers tagged: 107 · with a code link: 46 · where Syntology ran a sample: 10 (7 with a run with no instrument failure, 3 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (10 of 107 tagged: 7 with a run with no instrument failure, 3 where every run was a failure of Syntology's instrument)
Page 1 of 2: papers 1 to 100 of 107, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
25 Sep 2020 7 repositories listed Syntology 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 3 harvested samples) · 1 pointer-only (licence)robosuite is a simulation framework for robot learning powered by the MuJoCo physics engine.
-
8 Dec 2022 3 repositories listedThis work addresses the problem of generating 3D holistic body motions from human speech.
-
22 Aug 2022 3 repositories listedOn the other hand, all synthetic motion is found to be vastly less appropriate for the speech than the original motion-capture recordings.
-
24 Aug 2023 2 repositories listed Syntology community repositories only · 4 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 0 violated, 1 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 4 harvested samples) · 3 pointer-only (licence)The effect of the interlocutor is even more subtle, with submitted systems at best performing barely above chance.
-
4 Sep 2020 2 repositories listedIn this paper, we present an automatic gesture generation model that uses the multimodal context of speech text, audio, and speaker identity to reliably generate gestures.
-
10 Jun 2019 2 repositories listedSpecifically, we perform cross-modal translation from "in-the-wild'' monologue speech of a single speaker to their hand and arm motion.
-
3 Jul 2025 1 repository listedAlong with the explosion of large language models, improvements in speech synthesis, advancements in hardware, and the evolution of computer graphics, the current bottleneck in creating digital humans lies in generating…
-
21 May 2025 1 repository listedTo address this gap, we introduce Intentional-Gesture, a novel framework that casts gesture generation as an intention-reasoning task grounded in high-level communicative functions.
-
15 Feb 2025 1 repository listedGenerating expressive and diverse human gestures from audio is crucial in fields like human-computer interaction, virtual reality, and animation.
-
31 Jan 2025 1 repository listedTo overcome the suboptimal performance of flow matching baseline, we propose latent shortcut learning and beta distribution time stamp sampling during training to enhance gesture synthesis quality and accelerate…
-
9 Dec 2024 1 repository listedTherefore, we present RAG-Gesture, a diffusion-based gesture generation approach that leverages Retrieval Augmented Generation (RAG) to produce natural-looking and semantically rich gestures.
-
1 Oct 2024 1 repository listedCurrent co-speech motion generation approaches usually focus on upper body gestures following speech contents only, while lacking supporting the elaborate control of synergistic full-body motion based on text prompts…
-
30 Jul 2024 1 repository listedHowever, employing a unified model to achieve various generation tasks with different condition modalities presents two main challenges: motion distribution drifts across different tasks (e. g., co-speech gestures and…
-
1 Jun 2024 1 repository listedOnce trained, AMUSE synthesizes 3D human gestures directly from speech with control over the expressed emotions and style by combining the content from the driving speech with the emotion and style of another speech…
-
26 Mar 2024 1 repository listedGestures play a key role in human communication.
-
14 Mar 2024 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)Gesture synthesis is a vital realm of human-computer interaction, with wide-ranging applications across various fields like film, robotics, and virtual reality.
-
31 Dec 2023 1 repository listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 2 honoured, 1 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 4 harvested samples) · 4 pointer-only (licence)We propose EMAGE, a framework to generate full-body human gestures from audio and masked gestures, encompassing facial, local body, hands, and global movements.
-
7 Dec 2023 1 repository listedOnce trained, AMUSE synthesizes 3D human gestures directly from speech with control over the expressed emotions and style by combining the content from the driving speech with the emotion and style of another speech…
-
17 Sep 2023 1 repository listed Syntology official (archive's flag): 8 ran · 8 ran (of which 6 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified (of 10 harvested samples) · 10 pointer-only (licence)While previous methods are able to generate speech rhythm-synchronized gestures, the semantic context of the speech is generally lacking in the gesticulations.
-
13 Sep 2023 1 repository listedThe automatic co-speech gesture generation draws much attention in computer animation.
-
8 Sep 2023 1 repository listedCreating a diverse and comprehensive dataset of hand gestures for dynamic human-machine interfaces in the automotive domain can be challenging and time-consuming.
-
29 Aug 2023 1 repository listedCo-speech gesture generation is crucial for automatic digital avatar animation.
-
30 May 2023 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified (of 3 harvested samples) · 3 pointer-only (licence)In this work, we propose EmotionGesture, a novel framework for synthesizing vivid and diverse emotional co-speech 3D gestures from audio.
-
18 May 2023 1 repository listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 5 harvested samples) · 5 pointer-only (licence)Levenshtein distance based on audio quantization as a similarity metric of corresponding speech of gestures helps match more appropriate gestures with speech, and solves the alignment problem of speech and gestures well.
-
2 May 2023 1 repository listedBy learning the mapping of a latent space representation as opposed to directly mapping it to a vector representation, this framework facilitates the generation of highly realistic and expressive gestures that closely…
-
26 Mar 2023 1 repository listedWe leverage the power of the large-scale Contrastive-Language-Image-Pre-training (CLIP) model and present a novel CLIP-guided mechanism that extracts efficient style representations from multiple input modalities, such…
-
16 Mar 2023 1 repository listedIn this work, we propose a novel diffusion-based framework, named Diffusion Co-Speech Gesture (DiffGesture), to effectively capture the cross-modal audio-to-gesture associations and preserve temporal coherence for…
-
17 Nov 2022 1 repository listedDiffusion models have experienced a surge of interest as highly expressive yet efficiently trainable probabilistic models.
-
4 Oct 2022 1 repository listedWe present a novel co-speech gesture synthesis method that achieves convincing results both on the rhythm and semantics.
-
15 Sep 2022 1 repository listedIn a series of experiments, we first demonstrate the flexibility and generalizability of our model to new speakers and styles.
-
29 Aug 2022 1 repository listed
-
25 Aug 2022 1 repository listed
-
24 Mar 2022 1 repository listed
-
BEAT: A Large-Scale Semantic and Emotional Multi-Modal Dataset for Conversational Gestures Synthesis10 Mar 2022 1 repository listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples) · 3 pointer-only (licence)
-
1 Jan 2022 1 repository listed
-
29 Sep 2021 1 repository listed
-
18 Aug 2021 1 repository listed Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 7 harvested samples)
-
31 Jul 2021 1 repository listed
-
5 Jul 2021 1 repository listed
-
24 Feb 2021 1 repository listed
-
18 Nov 2020 1 repository listed
-
1 Oct 2020 1 repository listed
-
24 Jul 2020 1 repository listed
-
16 Jul 2020 1 repository listed
-
25 May 2020 1 repository listed
-
25 Jan 2020 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)
-
M3G: Multi-Granular Gesture Generator for Audio-Driven Full-Body Human Motion Synthesis13 May 2025 0 repositories listed
-
Inter-Diffusion Generation Model of Speakers and Listeners for Effective Communication8 May 2025 0 repositories listed
-
Co³Gesture: Towards Coherent Concurrent Co-speech 3D Gesture Generation with Interactive Diffusion3 May 2025 0 repositories listed
-
12 Apr 2025 0 repositories listed
-
EasyGenNet: An Efficient Framework for Audio-Driven Gesture Video Generation Based on Diffusion Model11 Apr 2025 0 repositories listed
-
Audio-driven Gesture Generation via Deviation Feature in the Latent Space27 Mar 2025 0 repositories listed
-
SARGes: Semantically Aligned Reliable Gesture Generation via Intent Chain26 Mar 2025 0 repositories listed
-
DIDiffGes: Decoupled Semi-Implicit Diffusion Models for Real-time Gesture Generation from Speech21 Mar 2025 0 repositories listed
-
Large Language Models for Virtual Human Gesture Selection18 Mar 2025 0 repositories listed
-
MAG: Multi-Modal Aligned Autoregressive Co-Speech Gesture Generation without Vector Quantization18 Mar 2025 0 repositories listed
-
Streaming Generation of Co-Speech Gestures via Accelerated Rolling Diffusion13 Mar 2025 0 repositories listed
-
HOP: Heterogeneous Topology-based Multimodal Entanglement for Co-Speech Gesture Generation3 Mar 2025 0 repositories listed
-
11 Feb 2025 0 repositories listed
-
EMO2: End-Effector Guided Audio-Driven Avatar Video Generation18 Jan 2025 0 repositories listed
-
Co-Speech Gesture Video Generation with Implicit Motion-Audio Entanglement1 Jan 2025 0 repositories listed
-
21 Dec 2024 0 repositories listed
-
The Language of Motion: Unifying Verbal and Non-verbal Language of 3D Human Motion13 Dec 2024 0 repositories listed
-
DiM-Gestor: Co-Speech Gesture Generation with Adaptive Layer Normalization Mamba-223 Nov 2024 0 repositories listed
-
Conditional GAN for Enhancing Diffusion Models in Efficient and Authentic Global Gesture Generation from Audios27 Oct 2024 0 repositories listed
-
Large Body Language Models21 Oct 2024 0 repositories listed
-
Emphasizing Semantic Consistency of Salient Posture for Speech-Driven Gesture Generation17 Oct 2024 0 repositories listed
-
ExpGest: Expressive Speaker Generation Using Diffusion Model and Hybrid Audio-Text Guidance12 Oct 2024 0 repositories listed
-
Towards a GENEA Leaderboard -- an Extended, Living Benchmark for Evaluating and Advancing Conversational Motion Synthesis8 Oct 2024 0 repositories listed
-
LLM Gesticulator: Leveraging Large Language Models for Scalable and Controllable Co-Speech Gesture Synthesis6 Oct 2024 0 repositories listed
-
MM-Conv: A Multi-modal Conversational Dataset for Virtual Humans30 Sep 2024 0 repositories listed
-
2D or not 2D: How Does the Dimensionality of Gesture Representation Affect 3D Co-Speech Gesture Generation?16 Sep 2024 0 repositories listed
-
Incorporating Spatial Awareness in Data-Driven Gesture Generation for Virtual Agents7 Aug 2024 0 repositories listed
-
MDT-A2G: Exploring Masked Diffusion Transformers for Co-Speech Gesture Generation6 Aug 2024 0 repositories listed
-
DiM-Gesture: Co-Speech Gesture Generation with Adaptive Layer Normalization Mamba-2 framework1 Aug 2024 0 repositories listed
-
Investigating the impact of 2D gesture representation on co-speech gesture generation21 Jun 2024 0 repositories listed
-
CoCoGesture: Toward Coherent Co-speech 3D Gesture Generation in the Wild27 May 2024 0 repositories listed
-
LLAniMAtion: LLAMA Driven Gesture Animation13 May 2024 0 repositories listed
-
Bridge to Non-Barrier Communication: Gloss-Prompted Fine-grained Cued Speech Gesture Generation with Diffusion Model30 Apr 2024 0 repositories listed
-
Speech-driven Personalized Gesture Synthetics: Harnessing Automatic Fuzzy Feature Inference16 Mar 2024 0 repositories listed
-
DiffSHEG: A Diffusion-Based Approach for Real-Time Speech-driven Holistic 3D Expression and Gesture Generation9 Jan 2024 0 repositories listed
-
Freetalker: Controllable Speech and Text-Driven Gesture Generation Based on Diffusion Models for Enhanced Speaker Naturalness7 Jan 2024 0 repositories listed
-
Chain of Generation: Multi-Modal Gesture Synthesis via Cascaded Conditional Control26 Dec 2023 0 repositories listed
-
Weakly-Supervised Emotion Transition Learning for Diverse 3D Co-speech Gesture Generation29 Nov 2023 0 repositories listed
-
Large language models in textual analysis for gesture selection4 Oct 2023 0 repositories listed
-
Speech-Gesture GAN: Gesture Generation for Robots and Embodied Agents17 Sep 2023 0 repositories listed
-
Diffusion-Based Co-Speech Gesture Generation Using Joint Text and Audio Representation11 Sep 2023 0 repositories listed
-
Audio is all in one: speech-driven gesture synthetics using WavLM pre-trained model11 Aug 2023 0 repositories listed
-
EMoG: Synthesizing Emotive Co-speech 3D Gesture with Diffusion Model20 Jun 2023 0 repositories listed
-
MPE4G: Multimodal Pretrained Encoder for Co-Speech Gesture Generation25 May 2023 0 repositories listed
-
GesGPT: Speech Gesture Synthesis With Text Parsing from ChatGPT23 Mar 2023 0 repositories listed
-
Evaluating gesture generation in a large-scale open challenge: The GENEA Challenge 202215 Mar 2023 0 repositories listed
-
Audio2Gestures: Generating Diverse Gestures from Audio17 Jan 2023 0 repositories listed
-
A Comprehensive Review of Data-Driven Co-Speech Gesture Generation13 Jan 2023 0 repositories listed
-
Continual Learning for Personalized Co-speech Gesture Generation1 Jan 2023 0 repositories listed
-
Zero-Shot Style Transfer for Gesture Animation driven by Text and Speech using Adversarial Disentanglement of Multimodal Style Encoding3 Aug 2022 0 repositories listed
-
Low-Resource Adaptation for Personalized Co-Speech Gesture Generation1 Jan 2022 0 repositories listed
-
15 Aug 2021 0 repositories listed
-
Multimodal analysis of the predictability of hand-gesture properties12 Aug 2021 0 repositories listed
-
Speech2Properties2Gestures: Gesture-Property Prediction as a Tool for Generating Representational Gestures from Speech28 Jun 2021 0 repositories listed
Syntology lines on 10 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections