Datasets › BEAT2

BEAT2 (BEAT-SMPLX-FLAME)

Introduced by Haiyang Liu et al. in EMAGE: Towards Unified Holistic Co-Speech Gesture Generation via Expressive Masked Audio Gesture Modeling31 Dec 2023 archive 2025-07-28

We propose EMAGE, a framework to generate full-body human gestures from audio and masked gestures, encompassing facial, local body, hands, and global movements. To achieve this, we first introduce BEAT2 (BEAT-SMPLX-FLAME), a new mesh-level holistic co-speech dataset. BEAT2 combines MoShed SMPLX body with FLAME head parameters and further refines the modeling of head, neck, and finger movements, offering a community-standardized, high-quality 3D motion captured dataset. EMAGE leverages masked body gesture priors during training to boost inference performance. It involves a Masked Audio Gesture Transformer, facilitating joint training on audio-to-gesture generation and masked gesture reconstruction to effectively encode audio and body gesture hints. Encoded body hints from masked gestures are then separately employed to generate facial and body movements. Moreover, EMAGE adaptively merges speech features from the audio's rhythm and content and utilizes four compositional VQ-VAEs to enhance the results' fidelity and diversity. Experiments demonstrate that EMAGE generates holistic gestures with state-of-the-art performance and is flexible in accepting predefined spatial-temporal gesture inputs, generating complete, audio-synchronized results.

Benchmarks archive 2025-07-28

All 2 leaderboards whose dataset resolves to this page shown (sort by any header). "First row" is the archive's own first row at snapshot, in the archive's row order; nothing here re-ranks and metric direction is not asserted.

Syntology 1 newer paper was judged to report on this dataset's benchmarks so far (Claude Sonnet 4.5, reading their tables: a model's judgement, not a result; 0 placed): 3D Face Animation · BEAT2 (1 judged, 0 placed). Drawn from 8,886 of 18,637 newer papers checked so far (733 more had nothing to place; 41 could not be read).

Papers archive 2025-07-28

papers with a benchmark row: 16 · with a code link: 12 · where Syntology ran a sample: 5 (4 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument) Syntology

Show: all papers with a benchmark rowonly where code ran (5 of 16 with a benchmark row: 4 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument)

Page 1 of 1: papers 1 to 16 of 16 with a leaderboard row on this dataset's benchmarks, newest first by the archive's date (ties by slug; undated papers last).

The archive's own "papers using this dataset" list was never published, so this is the benchmark-backed subset, not that list; the archive's count for this dataset is 23. The Syntology column is from Syntology's graph, stated per sample; it is not part of any archive number. A line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified” (C is a part of N, never taken away from it); the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. When the archive marks a repository official for the paper, the cell starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record.

PaperCodeResultsDateSamples run Syntology
Intentional Gesture: Deliver Your Intentions with Gestures for Speech 1 1 21 May 2025 not harvested
EchoMask: Speech-Queried Attention-based Mask Modeling for Holistic Co-Speech Motion Generation 0 1 12 Apr 2025 not harvested
Contextual Gesture: Co-Speech Gesture Video Generation through Context-aware Gesture Representation 0 1 11 Feb 2025 not harvested
GestureLSM: Latent Shortcut based Co-Speech Gesture Generation with Spatial-Temporal Modeling 1 1 31 Jan 2025 not harvested
SemTalk: Holistic Co-speech Motion Generation with Frame-level Semantic Emphasis 0 1 21 Dec 2024 not harvested
Enabling Synergistic Full-Body Control in Prompt-Based Co-Speech Motion Generation 1 1 1 Oct 2024 not harvested
MambaTalk: Efficient Holistic Gesture Synthesis with Selective State Space Models 1 2 14 Mar 2024 official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified
EMAGE: Towards Unified Holistic Co-Speech Gesture Generation via Expressive Masked Audio Gesture Modeling 1 1 31 Dec 2023 official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 2 honoured, 1 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (4 pointer-only for licence)
CodeTalker: Speech-Driven 3D Facial Animation with Discrete Motion Prior 1 1 6 Jan 2023 official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 1 honoured, 1 violated, 4 with no contract checked; 2 where Syntology's instrument failed) · 5 unverified
Generating Holistic 3D Human Motion from Speech 3 2 8 Dec 2022 not harvested
Learning Hierarchical Cross-Modal Association for Co-Speech Gesture Generation 1 1 24 Mar 2022 not harvested
BEAT: A Large-Scale Semantic and Emotional Multi-Modal Dataset for Conversational Gestures Synthesis 1 1 10 Mar 2022 official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (3 pointer-only for licence)
FaceFormer: Speech-Driven 3D Facial Animation with Transformers 1 1 10 Dec 2021 official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 0 violated, 1 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (2 pointer-only for licence)
Learning Speech-driven 3D Conversational Gestures from Video 0 2 13 Feb 2021 not harvested
Speech Gesture Generation from the Trimodal Context of Text, Audio, and Speaker Identity 2 1 4 Sep 2020 not harvested
Learning Individual Styles of Conversational Gesture 2 1 10 Jun 2019 not harvested

Dataset loaders archive 2025-07-28

No loader listed in the archive.

Tasks archive 2025-07-28

License archive 2025-07-28

CC-BY-NC-4.0

Modalities archive 2025-07-28

Languages archive 2025-07-28

Variants archive 2025-07-28

  • BEAT2

1 variant name, as the archive lists them.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections