Social Structure Matters in 3D Human-Human Interaction Generation
- URL: http://arxiv.org/abs/2606.24255v1
- Date: Tue, 23 Jun 2026 07:43:13 GMT
- Title: Social Structure Matters in 3D Human-Human Interaction Generation
- Abstract summary: We formulate HHI generation as a social structure modeling and grounding problem.<n>The model must first infer how an interaction unfolds and how the two actors coordinate their roles.<n>Our framework produces 3D HHI with improved phase consistency, role alignment, and partner-aware coordination.
- Score: 70.08319433924056
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Although text-to-motion generation has achieved strong progress in synthesizing realistic single-person motions from language, extending it to text-driven 3D human-human interaction (HHI) remains non-trivial, as HHI requires modeling the underlying \textbf{social structure} that governs phase progression, actor roles, and inter-actor coordination. In this paper, we formulate HHI generation as a social structure modeling and grounding problem: the model must first infer how an interaction unfolds and how the two actors coordinate their roles, and then realize this structure as continuous, physically plausible, and partner-aware 3D motion. To study how such structure should be modeled, we first examine the capability boundary of large language models (LLMs) for HHI generation. Our analysis shows that LLMs can \textit{think} by recovering phase decompositions and partner-aware roles, but cannot directly \textit{move}, as they fail to generate dynamic, physically plausible, and interaction-aware motion. This motivates our planner-executor paradigm, \textbf{Think with LLM, Move with Motion Skill}. The LLM planner converts implicit interaction semantics into motion-aligned social supervision by decomposing interactions into phases, assigning partner-aware actor roles, and aligning them with motion sequence. The motion executor then grounds the planned social structure into coordinated two-person motion by adapting a pretrained solo motion model with LoRA, previous-phase self-conditioning, and ego-relative partner conditioning. Together, our Solo-to-Social framework bridges social organization and motion realization, producing 3D HHI with improved phase consistency, role alignment, and partner-aware coordination.
Related papers
- EgoMotion: Hierarchical Reasoning and Diffusion for Egocentric Vision-Language Motion Generation [74.07852250099559]
We investigate Egocentric Vision-Language (Ego-VL) motion generation.<n>This task requires synthesizing 3D human motion conditioned jointly on first-person visual observations and natural language instructions.<n>We propose a hierarchical generative framework textbfEgoMotion to address this challenge.
arXiv Detail & Related papers (2026-04-21T05:31:06Z) - SocialMirror: Reconstructing 3D Human Interaction Behaviors from Monocular Videos with Semantic and Geometric Guidance [49.69016078147708]
Accurately reconstructing human behavior in close-interaction scenarios is crucial for enabling realistic virtual interactions in augmented reality.<n>We propose SocialMirror, a diffusion-based framework that integrates semantic and geometric cues to effectively address these issues.<n>SocialMirror achieves state-of-the-art performance in reconstructing interactive human meshes, demonstrating strong generalization across unseen datasets and in-the-wild scenarios.
arXiv Detail & Related papers (2026-04-15T07:41:52Z) - HO-Flow: Generalizable Hand-Object Interaction Generation with Latent Flow Matching [113.81911881001905]
HO-Flow is a framework for synthesizing realistic hand-object motion sequences from texts and canoncial 3D objects.<n>It first employs an interaction-aware variational autoencoder to encode sequences of hand and object motions into a unified latent manifold.<n>It then leverages a masked flow matching model that combines auto-regressive temporal reasoning with continuous latent generation.
arXiv Detail & Related papers (2026-04-12T22:06:11Z) - Real-Time Synchronized Interaction Framework for Emotion-Aware Humanoid Robots [0.5156484100374058]
We present a real-time framework for NAO robots that synchronizes speech prosody with full-body gestures.<n>By enabling seamless sensorimotor coordination, this framework advances the deployment of context-aware social robots.
arXiv Detail & Related papers (2026-01-24T03:53:09Z) - InterSyn: Interleaved Learning for Dynamic Motion Synthesis in the Wild [65.29569330744056]
We present Interleaved Learning for Motion Synthesis (InterSyn), a novel framework that targets the generation of realistic interaction motions.<n>InterSyn employs an interleaved learning strategy to capture the natural, dynamic interactions and nuanced coordination inherent in real-world scenarios.
arXiv Detail & Related papers (2025-08-14T03:00:06Z) - UniHM: Universal Human Motion Generation with Object Interactions in Indoor Scenes [26.71077287710599]
We propose UniHM, a unified motion language model that leverages diffusion-based generation for scene-aware human motion.<n>UniHM is the first framework to support both Text-to-Motion and Text-to-Human-Object Interaction (HOI) in complex 3D scenes.<n>Our approach introduces three key contributions: (1) a mixed-motion representation that fuses continuous 6DoF motion with discrete local motion tokens to improve motion realism; (2) a novel Look-Up-Free Quantization VAE that surpasses traditional VQ-VAEs in both reconstruction accuracy and generative performance; and (3) an enriched version of
arXiv Detail & Related papers (2025-05-19T07:02:12Z) - Human-Object Interaction via Automatically Designed VLM-Guided Motion Policy [30.43930233035367]
We introduce the first unified physics-based HO framework that leverages Vision-Language Models (VLMs)<n>We introduce VLM-Guided Relative Movement Dynamics (RMD), a fine-temporal bipartite motion representation that automatically constructs goal states and reward functions for reinforcement learning.<n>To support our methodology, we present Interplay, a novel dataset with thousands of long-horizon static and dynamic interaction plans.
arXiv Detail & Related papers (2025-03-24T05:18:04Z) - Jointly Understand Your Command and Intention:Reciprocal Co-Evolution between Scene-Aware 3D Human Motion Synthesis and Analysis [80.50342609047091]
Scene-aware text-to-human synthesis generates diverse indoor motion samples from the same textual description.<n>We propose a cascaded generation strategy that factorizes text-driven scene-specific human motion generation into three stages.<n>We jointly improve realistic human motion synthesis and robust human motion analysis in 3D scenes.
arXiv Detail & Related papers (2025-03-01T06:56:58Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.