GDPO-Listener: Expressive Interactive Head Generation via Auto-Regressive Flow Matching and Group reward-Decoupled Policy Optimization

• URL: http://arxiv.org/abs/2603.25020v1
• Date: Thu, 26 Mar 2026 04:36:27 GMT
• Title: GDPO-Listener: Expressive Interactive Head Generation via Auto-Regressive Flow Matching and Group reward-Decoupled Policy Optimization
• Abstract summary: GDPO-Listener is a novel framework that achieves highly expressive speaking and listening motion generation.<n>We introduce an Auto-Regressive Flow Matching architecture enabling stable supervised learning.<n>By isolating reward normalization across distinct FLAME parameter groups, GDPO explicitly incentivizes high variance expressive generations.
• Score: 5.5806589487812746
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generating realistic 3D head motion for dyadic interactions is a significant challenge in virtual human synthesis. While recent methods achieve impressive results with speaking heads, they frequently suffer from the `Regression-to-the-Mean' problem in listener motions, collapsing into static faces, and lack the parameter space for complex nonverbal motions. In this paper, we propose GDPO-Listener, a novel framework that achieves highly expressive speaking and listening motion generation. First, we introduce an Auto-Regressive Flow Matching architecture enabling stable supervised learning. Second, to overcome kinematic stillness, we apply the Group reward-Decoupled Policy Optimization (GDPO). By isolating reward normalization across distinct FLAME parameter groups, GDPO explicitly incentivizes high variance expressive generations. Finally, we enable explicit semantic text control for customizable responses. Extensive evaluations across the Seamless Interaction and DualTalk datasets demonstrate superior performance compared to existing baselines on long-term kinematic variance, visual expressivity and semantic controllability.

Related papers

• Inter-X++: A Comprehensive Benchmark for Multimodal Human-Human Interaction Analysis [55.78822230833247]
  We present Inter-X++, a comprehensive and large-scale benchmark designed to empower versatile HHI analysis.<n>Inter-X++ provides 11,388 high-fidelity interaction sequences and over 8.1M frames, featuring precise whole-body movements and detailed finger articulations.<n>We propose OpenHHI, a single and unified HHI representation and modeling framework that jointly optimize interaction reconstruction and semantic understanding.
  arXiv  Detail & Related papers  (2026-08-20T17:51:48Z)
• SubtleTalk: Generating Controllable Weakly-correlated Facial Dynamics for 3D Talking Heads via Residual Flow Matching [21.891013914273714]
  Audio-driven 3D facial animation aims to synthesize realistic and temporally coherent motions from speech.<n>We present SubtleTalk, a framework for generating natural and controllable weakly correlated facial dynamics.<n>Our method significantly improves the realism and diversity of weakly correlated facial dynamics while preserving accurate lip synchronization.
  arXiv  Detail & Related papers  (2026-08-03T11:28:19Z)
• SELF-EMO: Emotional Self-Evolution from Recognition to Consistent Expression [13.896538941019722]
  We propose SELF-EMO, a self-evolution framework grounded in the hypothesis that better emotion prediction leads to more consistent emotional responses.<n>We introduce two auxiliary tasks, emotional understanding and emotional expression, and design a role-based self-play paradigm where the model acts as both an emotion recognizer and a dialogue responder.<n> Experiments on IEMOCAP, MELD, and EmoryNLP show that SELF-EMO achieves state-of-the-art performance, improving accuracy by +6.33% on Qwen3-4B and +8.54% on Qwen3-8B.
  arXiv  Detail & Related papers  (2026-04-20T09:27:40Z)
• InterDyad: Interactive Dyadic Speech-to-Video Generation by Querying Intermediate Visual Guidance [20.740979380270126]
  We propose InterDyad, a synthesis framework for interactive dyadic dynamics.<n>We first design an Interactivity that achieves video reenactment based on identity-agnostic motion priors extracted from reference videos.<n>By leveraging a Multimodal Large Language Model (MLLM), our framework is able to distill linguistic intent from audio to dictate the precise timing and appropriateness of reactions.<n> Comprehensive experiments demonstrate that InterDyad significantly outperforms state-of-the-art methods in producing natural and contextually grounded two-person interactions.
  arXiv  Detail & Related papers  (2026-03-24T12:27:52Z)
• MIBURI: Towards Expressive Interactive Gesture Synthesis [62.45332399212876]
  Embodied Conversational Agents (ECAs) aim to emulate human face-to-face interaction through speech, gestures, and facial expressions.<n>Existing solutions for ECAs produce rigid, low-diversity motions that are unsuitable for human-like interaction.<n>We present MIBURI, the first online, causal framework for generating expressive full-body gestures and facial expressions synchronized with real-time spoken dialogue.
  arXiv  Detail & Related papers  (2026-03-03T18:59:51Z)
• HINT: Hierarchical Interaction Modeling for Autoregressive Multi-Human Motion Generation [55.73037290387896]
  We introduce HINT, the first autoregressive framework for multi-human motion generation with Hierarchical INTeraction modeling in diffusion.<n>First, HINT leverages a disentangled motion representation within a canonicalized latent space, decoupling local motion semantics from inter-person interactions.<n>Second, HINT adopts a sliding-window strategy for efficient online generation, and aggregates local within-window and global cross-window conditions to capture past human history, inter-person dependencies, and align with text guidance.
  arXiv  Detail & Related papers  (2026-01-28T08:47:23Z)
• Interact2Ar: Full-Body Human-Human Interaction Generation via Autoregressive Diffusion Models [80.28579390566298]
  We introduce Interact2Ar, a text-conditioned autoregressive diffusion model for generating full-body, human-human interactions.<n>Hand kinematics are incorporated through dedicated parallel branches, enabling high-fidelity full-body generation.<n>Our model enables a series of downstream applications, including temporal motion composition, real-time adaptation to disturbances, and extension beyond dyadic to multi-person scenarios.
  arXiv  Detail & Related papers  (2025-12-22T18:59:50Z)
• ARIG: Autoregressive Interactive Head Generation for Real-time Conversations [15.886402427095515]
  Face-to-face communication, as a common human activity, motivates the research on interactive head generation.<n>Previous clip-wise generation paradigm or explicit listener/speaker generator-switching methods have limitations in future signal acquisition.<n>We propose an autoregressive (AR) based frame-wise framework called ARIG to realize the real-time generation with better interaction realism.
  arXiv  Detail & Related papers  (2025-07-01T06:38:14Z)
• ReCoM: Realistic Co-Speech Motion Generation with Recurrent Embedded Transformer [58.49950218437718]
  We present ReCoM, an efficient framework for generating high-fidelity and generalizable human body motions synchronized with speech.<n>The core innovation lies in the Recurrent Embedded Transformer (RET), which integrates Dynamic Embedding Regularization (DER) into a Vision Transformer (ViT) core architecture.<n>To enhance model robustness, we incorporate the proposed DER strategy, which equips the model with dual capabilities of noise resistance and cross-domain generalization.
  arXiv  Detail & Related papers  (2025-03-27T16:39:40Z)
• Auto-Regressive Diffusion for Generating 3D Human-Object Interactions [5.587507490937267]
  Key challenge in HOI generation is maintaining interaction consistency in long sequences.<n>We propose an autoregressive diffusion model (ARDHOI) that predicts the next continuous token.<n>Our model has been evaluated on the OMOMO and BEHAVE datasets.
  arXiv  Detail & Related papers  (2025-03-21T02:25:59Z)
• Lifting Scheme-Based Implicit Disentanglement of Emotion-Related Facial Dynamics in the Wild [3.3905929183808796]
  In-the-wild dynamic facial expression recognition (DFER) encounters a significant challenge in recognizing emotion-related expressions.<n>We propose a novel Implicit Facial Dynamics Disentanglement framework (IFDD)<n>IFDD disentangles emotion-related dynamic information from emotion-irrelevant global context in an implicit manner.
  arXiv  Detail & Related papers  (2024-12-17T18:45:53Z)
• DiffSHEG: A Diffusion-Based Approach for Real-Time Speech-driven Holistic 3D Expression and Gesture Generation [72.85685916829321]
  DiffSHEG is a Diffusion-based approach for Speech-driven Holistic 3D Expression and Gesture generation with arbitrary length. By enabling the real-time generation of expressive and synchronized motions, DiffSHEG showcases its potential for various applications in the development of digital humans and embodied agents.
  arXiv  Detail & Related papers  (2024-01-09T11:38:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.