Browse State-of-the-Art › Talking Face Generation
Talking Face Generation
43 papers with code · 2 benchmarks · 6 datasets archive 2025-07-28
Talking face generation aims to synthesize a sequence of face images that correspond to given speech semantics
( Image credit: Talking Face Generation by Adversarially Disentangled Audio-Visual Representation )
Description from the archive archive 2025-07-28.
Benchmarks archive 2025-07-28
2 leaderboard tables shown for this task, 2 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.
| Dataset | Best model (first row in archive order) | Paper | Code | Syntology | Compare |
|---|---|---|---|---|---|
| CREMA-D (1 row) | EmoGen | Emotionally Enhanced Talking Face Generation | code | — | Compare |
| LRW (1 row) | LipGAN | Towards Automatic Face-to-Face Translation | code | Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified | Compare |
Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. Each line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the row's paper, the line starts with that repository's state (“official” is the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. A dash means no Syntology record for that paper, not a recorded non-run. Each “ran” line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
6 datasets whose archive record lists this task, ordered by the archive's paper count.
Subtasks archive 2025-07-28
2 subtasks in the archive's task tree.
Parent tasks archive 2025-07-28
Papers archive 2025-07-28
archive papers tagged: 110 · with a code link: 43 · where Syntology ran a sample: 19 (18 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (19 of 110 tagged: 18 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument)
Page 1 of 2: papers 1 to 100 of 110, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
23 Aug 2020 4 repositories listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 2 harvested samples) · 2 pointer-only (licence)However, they fail to accurately morph the lip movements of arbitrary identities in dynamic, unconstrained talking face videos, resulting in significant parts of the video being out-of-sync with the new audio.
-
27 Apr 2020 3 repositories listed Syntology 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 2 pointer-only (licence)We present a method that generates expressive talking heads from a single facial image with audio as the only input.
-
22 Nov 2022 2 repositories listed Syntology 9 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 0 violated, 6 with no contract checked; 3 where Syntology's instrument failed) · 2 unverified (of 11 harvested samples) · 3 pointer-only (licence)While dynamic Neural Radiance Fields (NeRF) have shown success in high-fidelity 3D modeling of talking portraits, the slow training and inference speed severely obstruct their potential usage.
-
3 Jan 2025 1 repository listedSignificant progress has been made in talking-face video generation research; however, precise lip-audio synchronization and high visual quality remain challenging in editing lip shapes based on input audio.
-
18 Dec 2024 1 repository listedRecent advances in co-speech gesture and talking head generation have been impressive, yet most methods focus on only one of the two tasks.
-
9 Sep 2024 1 repository listedIn this paper, we propose the KFusion of Dual-Domain model, a robust model that generates landmarks from audio.
-
5 Jun 2024 1 repository listedAudio-driven talking face generation has garnered significant interest within the domain of digital human research.
-
26 Mar 2024 1 repository listedDeepfake is a technology dedicated to creating highly realistic facial images and videos under specific conditions, which has significant application potential in fields such as entertainment, movie production, digital…
-
16 Jan 2024 1 repository listed Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 1 honoured, 0 violated, 3 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 6 harvested samples)One-shot 3D talking portrait generation aims to reconstruct a 3D avatar from an unseen image, and then animate it with a reference video or audio to generate a talking portrait video.
-
12 Dec 2023 1 repository listedOur proposed GSmoothFace model mainly consists of the Audio to Expression Prediction (A2EP) module and the Target Adaptive Face Translation (TAFT) module.
-
11 Dec 2023 1 repository listedOur method, which we call NEUral Text to ARticulate Talk (NEUTART), is a talking face generator that uses a joint audiovisual feature space, as well as speech-informed 3D facial reconstructions and a lip-reading loss…
-
29 Nov 2023 1 repository listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified (of 4 harvested samples) · 4 pointer-only (licence)A lifelike talking head requires synchronized coordination of subject identity, lip movements, facial expressions, and head poses.
-
3 Nov 2023 1 repository listedOur rigorous experiments comprehensively highlight that our ground-breaking approach outpaces existing methods with considerable margins and delivers seamless, intelligible videos in person-generic and multilingual…
-
9 Oct 2023 1 repository listedFirst, FaceEncoder is used to obtain latent code by extracting features from the visual face information taken from the video source containing the face frame.
-
14 Sep 2023 1 repository listedIn particular, we propose a Fine-Grained Feature Fusion (FGFF) module to effectively capture fine texture feature information around teeth and surrounding regions, and use these features to fine-grain the feature map to…
-
15 May 2023 1 repository listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 6 harvested samples)Prior landmark characteristics of the speaker's face are employed to make the generated landmarks coincide with the facial outline of the speaker.
-
29 Mar 2023 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)To address the problem, we propose using a lip-reading expert to improve the intelligibility of the generated lip regions by penalizing the incorrect generation results.
-
21 Mar 2023 1 repository listedTo mitigate this, we build a talking face generation framework conditioned on a categorical emotion to generate videos with appropriate expressions, making them more realistic and convincing.
-
7 Mar 2023 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 1 where Syntology's instrument failed) · 8 unverified (of 10 harvested samples) · 10 pointer-only (licence)Different from previous works relying on multiple up-sample layers to directly generate pixels from latent embeddings, DINet performs spatial deformation on feature maps of reference images to better preserve…
-
31 Jan 2023 1 repository listed Syntology 3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified (of 4 harvested samples) · 2 pointer-only (licence)Generating photo-realistic video portrait with arbitrary speech audio is a crucial problem in film-making and virtual reality.
-
16 Jan 2023 1 repository listed Syntology official (archive's flag): 14 ran · 15 ran (of which 9 constructed an object rather than computing a result; 12 with no instrument failure: 0 honoured, 1 violated, 11 with no contract checked; 3 where Syntology's instrument failed) · 8 unverified (of 23 harvested samples) · 1 pointer-only (licence)In this paper, we introduce a novel self-supervised disentanglement framework to decouple pose and expression without 3DMMs and paired data, which consists of a motion editing module, a pose generator, and an expression…
-
3 Jan 2023 1 repository listed Syntology official: harvested, nothing ran · 0 ran · 2 unverified (of 2 harvested samples)In a nutshell, we aim to attain a speaking style from an arbitrary reference speaking video and then drive the one-shot portrait to speak with the reference speaking style and another piece of audio.
-
21 Sep 2022 1 repository listed Syntology official (archive's flag): 10 ran · 10 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 3 where Syntology's instrument failed) · 1 unverified (of 11 harvested samples) · 1 pointer-only (licence)In FNeVR, we design a 3D Face Volume Rendering (FVR) module to enhance the facial details for image rendering.
-
24 Jul 2022 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; the one sample that ran constructed an object rather than computing a result (of 1 harvested sample)Thus the facial radiance field can be flexibly adjusted to the new identity with few reference images.
-
1 Jun 2022 1 repository listedWe introduce the Merkel Podcast Corpus, an audio-visual-text corpus in German collected from 16 years of (almost) weekly Internet podcasts of former German chancellor Angela Merkel.
-
24 May 2022 1 repository listedWe introduce the Merkel Podcast Corpus, an audio-visual-text corpus in German collected from 16 years of (almost) weekly Internet podcasts of former German chancellor Angela Merkel.
-
8 Mar 2022 1 repository listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 1 where Syntology's instrument failed) · 4 unverified (of 8 harvested samples) · 1 pointer-only (licence)Our framework elevates the resolution of the synthesized talking face to 1024*1024 for the first time, even though the training dataset has a lower resolution.
-
22 Sep 2021 1 repository listedThe first stage is a deep neural network that extracts deep audio features along with a manifold projection to project the features to the target person's speech space.
-
18 Aug 2021 1 repository listed Syntology official (archive's flag): 3 ran · 3 ran (of which 3 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; every one of the 3 samples that ran constructed an object rather than computing a result (of 3 harvested samples) · 3 pointer-only (licence)In this paper, we propose a talking face generation method that takes an audio signal as input and a short target video clip as reference, and synthesizes a photo-realistic video of the target face with natural lip…
-
14 Jul 2021 1 repository listedHowever, the AR decoding manner generates current lip frame conditioned on frames generated previously, which inherently hinders the inference speed, and also has a detrimental effect on the quality of generated lip…
-
26 Jun 2021 1 repository listed
-
19 Jun 2021 1 repository listed
-
29 Apr 2021 1 repository listed
-
22 Apr 2021 1 repository listed Syntology official (archive's flag): 7 ran · 7 ran (of which 6 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified (of 10 harvested samples) · 10 pointer-only (licence)
-
20 Mar 2021 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; the one sample that ran constructed an object rather than computing a result (of 1 harvested sample)
-
21 Nov 2020 1 repository listed
-
8 Aug 2020 1 repository listed
-
1 Aug 2020 1 repository listed
-
11 Dec 2019 1 repository listed
-
8 May 2019 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)
-
29 Jul 2018 1 repository listed
-
20 Jul 2018 1 repository listed Syntology 4 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified (of 5 harvested samples) · 2 pointer-only (licence)
-
13 Apr 2018 1 repository listed
-
DisentTalk: Cross-lingual Talking Face Generation via Semantic Disentangled Diffusion Model24 Mar 2025 0 repositories listed
-
UniSync: A Unified Framework for Audio-Visual Synchronization20 Mar 2025 0 repositories listed
-
PC-Talk: Precise Facial Animation Control for Audio-Driven Talking Face Generation18 Mar 2025 0 repositories listed
-
Playmate: Flexible Control of Portrait Animation via 3D-Implicit Space Guided Diffusion11 Feb 2025 0 repositories listed
-
GLCF: A Global-Local Multimodal Coherence Analysis Framework for Talking Face Generation Detection18 Dec 2024 0 repositories listed
-
VQTalker: Towards Multilingual Talking Avatars through Facial Motion Tokenization13 Dec 2024 0 repositories listed
-
PortraitTalk: Towards Customizable One-Shot Audio-to-Talking Face Generation10 Dec 2024 0 repositories listed
-
Sonic: Shifting Focus to Global Audio Perception in Portrait Animation25 Nov 2024 0 repositories listed
-
MimicTalk: Mimicking a personalized and expressive 3D talking face in minutes9 Oct 2024 0 repositories listed
-
JEAN: Joint Expression and Audio-guided NeRF-based Talking Face Generation18 Sep 2024 0 repositories listed
-
StyleTalk++: A Unified Framework for Controlling the Speaking Styles of Talking Heads14 Sep 2024 0 repositories listed
-
SegTalker: Segmentation-based Talking Face Generation with Mask-guided Local Editing5 Sep 2024 0 repositories listed
-
High-fidelity and Lip-synced Talking Face Synthesis via Landmark-based Diffusion Model10 Aug 2024 0 repositories listed
-
Emotional Conversation: Empowering Talking Faces with Cohesive Expression, Gaze and Pose Generation12 Jun 2024 0 repositories listed
-
OpFlowTalker: Realistic and Natural Talking Face Generation via Optical Flow Guidance23 May 2024 0 repositories listed
-
Faces that Speak: Jointly Synthesising Talking Face and Speech from Text16 May 2024 0 repositories listed
-
SPEAK: Speech-Driven Pose and Emotion-Adjustable Talking Head Generation12 May 2024 0 repositories listed
-
SwapTalk: Audio-Driven Talking Face Generation with One-Shot Customization in Latent Space9 May 2024 0 repositories listed
-
Audio-Visual Speech Representation Expert for Enhanced Talking Face Video Generation and Evaluation7 May 2024 0 repositories listed
-
GSTalker: Real-time Audio-Driven Talking Face Generation via Deformable Gaussian Splatting29 Apr 2024 0 repositories listed
-
Superior and Pragmatic Talking Face Generation with Teacher-Student Framework26 Mar 2024 0 repositories listed
-
FlowVQTalker: High-Quality Emotional Talking Face Generation through Normalizing Flow and Quantization11 Mar 2024 0 repositories listed
-
Style2Talker: High-Resolution Talking Head Generation with Emotion Style and Art Style11 Mar 2024 0 repositories listed
-
G4G:A Generic Framework for High Fidelity Talking Face Generation with Fine-grained Intra-modal Alignment28 Feb 2024 0 repositories listed
-
AVI-Talking: Learning Audio-Visual Instructions for Expressive 3D Talking Face Generation25 Feb 2024 0 repositories listed
-
EmoSpeaker: One-shot Fine-grained Emotion-Controlled Talking Face Generation2 Feb 2024 0 repositories listed
-
EmoTalker: Emotionally Editable Talking Face Generation via Diffusion Model16 Jan 2024 0 repositories listed
-
DREAM-Talk: Diffusion-based Realistic Emotional Audio-driven Method for Single Image Talking Face Generation21 Dec 2023 0 repositories listed
-
VectorTalker: SVG Talking Face Generation with Progressive Vectorisation18 Dec 2023 0 repositories listed
-
FT2TF: First-Person Statement Text-To-Talking Face Generation9 Dec 2023 0 repositories listed
-
CP-EB: Talking Face Generation with Controllable Pose and Eye Blinking Embedding15 Nov 2023 0 repositories listed
-
ToonTalker: Cross-Domain Face Reenactment24 Aug 2023 0 repositories listed
-
VAST: Vivify Your Talking Avatar via Zero-Shot Expressive Facial Style Transfer9 Aug 2023 0 repositories listed
-
Audio-driven Talking Face Generation with Stabilized Synchronization Loss18 Jul 2023 0 repositories listed
-
FTFDNet: Learning to Detect Talking Face Video Manipulation with Tri-Modality Interaction8 Jul 2023 0 repositories listed
-
Instruct-NeuralTalker: Editing Audio-Driven Talking Radiance Fields with Instructions19 Jun 2023 0 repositories listed
-
Exploring Phonetic Context-Aware Lip-Sync For Talking Face Generation31 May 2023 0 repositories listed
-
CPNet: Exploiting CLIP-based Attention Condenser and Probability Map Guidance for High-fidelity Talking Face Generation23 May 2023 0 repositories listed
-
High-fidelity Generalized Emotional Talking Face Generation with Multi-modal Emotion Space Learning4 May 2023 0 repositories listed
-
Multimodal-driven Talking Face Generation via a Unified Diffusion-based Generator4 May 2023 0 repositories listed
-
GeneFace++: Generalized and Stable Real-Time Audio-Driven 3D Talking Face Generation1 May 2023 0 repositories listed
-
Audio-Driven Talking Face Generation with Diverse yet Realistic Facial Animations18 Apr 2023 0 repositories listed
-
That's What I Said: Fully-Controllable Talking Face Generation6 Apr 2023 0 repositories listed
-
A Unified Compression Framework for Efficient Speech-Driven Talking-Face Generation2 Apr 2023 0 repositories listed
-
DAE-Talker: High Fidelity Speech-Driven Talking Face Generation with Diffusion Autoencoder30 Mar 2023 0 repositories listed
-
UniFLG: Unified Facial Landmark Generator from Text or Speech28 Feb 2023 0 repositories listed
-
Diffused Heads: Diffusion Models Beat GANs on Talking-Face Generation6 Jan 2023 0 repositories listed
-
EMMN: Emotional Motion Memory Network for Audio-driven Emotional Talking Face Generation1 Jan 2023 0 repositories listed
-
LipFormer: High-Fidelity and Generalizable Talking Face Generation With a Pre-Learned Facial Codebook1 Jan 2023 0 repositories listed
-
Emotional Talking Faces: Making Videos More Expressive and Realistic13 Dec 2022 0 repositories listed
-
Memories are One-to-Many Mapping Alleviators in Talking Face Generation9 Dec 2022 0 repositories listed
-
SyncTalkFace: Talking Face Generation with Precise Lip-Syncing via Audio-Lip Memory2 Nov 2022 0 repositories listed
-
Taiwanese-Accented Mandarin and English Multi-Speaker Talking-Face Synthesis System1 Nov 2022 0 repositories listed
-
StableFace: Analyzing and Improving Motion Stability for Talking Face Generation29 Aug 2022 0 repositories listed
-
Cut Inner Layers: A Structured Pruning Strategy for Efficient U-Net GANs29 Jun 2022 0 repositories listed
-
EAMM: One-Shot Emotional Talking Face via Audio-Based Emotion-Aware Motion Model30 May 2022 0 repositories listed
-
One-Shot Face Reenactment on Megapixels26 May 2022 0 repositories listed
Syntology lines on 19 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections