ARFlow: Human Action-Reaction Flow Matching with Physical Guidance

• URL: http://arxiv.org/abs/2503.16973v2
• Date: Wed, 26 Mar 2025 08:43:09 GMT
• Title: ARFlow: Human Action-Reaction Flow Matching with Physical Guidance
• Authors: Wentao Jiang, Jingya Wang, Haotao Lu, Kaiyang Ji, Baoxiong Jia, Siyuan Huang, Ye Shi,
• Abstract summary: Action-Reaction Flow Matching is a novel framework that establishes direct action-to-reaction mappings.<n>Our approach introduces two key innovations: an x1-prediction method that directly outputs human motions instead of velocity fields, and a training-free, gradient-based physical guidance mechanism that effectively prevents body penetration artifacts during sampling.
• Score: 34.33083853308399
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Human action-reaction synthesis, a fundamental challenge in modeling causal human interactions, plays a critical role in applications ranging from virtual reality to social robotics. While diffusion-based models have demonstrated promising performance, they exhibit two key limitations for interaction synthesis: reliance on complex noise-to-reaction generators with intricate conditional mechanisms, and frequent physical violations in generated motions. To address these issues, we propose Action-Reaction Flow Matching (ARFlow), a novel framework that establishes direct action-to-reaction mappings, eliminating the need for complex conditional mechanisms. Our approach introduces two key innovations: an x1-prediction method that directly outputs human motions instead of velocity fields, enabling explicit constraint enforcement; and a training-free, gradient-based physical guidance mechanism that effectively prevents body penetration artifacts during sampling. Extensive experiments on NTU120 and Chi3D datasets demonstrate that ARFlow not only outperforms existing methods in terms of Fr\'echet Inception Distance and motion diversity but also significantly reduces body collisions, as measured by our new Intersection Volume and Intersection Frequency metrics.

Related papers

• HOI-Dyn: Learning Interaction Dynamics for Human-Object Motion Diffusion [11.26861317672778]
  We present HOI-Dyn, a novel framework that formulates HOI generation as a driver-responder system.<n>At the core of our method is a lightweight transformer-based interaction dynamics model.<n>Our approach not only enhances the quality of HOI generation but also establishes a feasible metric for evaluating the quality of generated interactions.
  arXiv  Detail & Related papers  (2025-07-02T14:13:48Z)
• Neural Network Reprogrammability: A Unified Theme on Model Reprogramming, Prompt Tuning, and Prompt Instruction [55.914891182214475]
  We introduce neural network reprogrammability as a unifying framework for model adaptation.<n>We present a taxonomy that categorizes such information manipulation approaches across four key dimensions.<n>We also analyze remaining technical challenges and ethical considerations.
  arXiv  Detail & Related papers  (2025-06-05T05:42:27Z)
• REWIND: Real-Time Egocentric Whole-Body Motion Diffusion with Exemplar-Based Identity Conditioning [95.07708090428814]
  We present REWIND, a one-step diffusion model for real-time, high-fidelity human motion estimation from egocentric image inputs. We introduce cascaded body-hand denoising diffusion, which effectively models the correlation between egocentric body and hand motions. We also propose a novel identity conditioning method based on a small set of pose exemplars of the target identity, which further enhances motion estimation quality.
  arXiv  Detail & Related papers  (2025-04-07T11:44:11Z)
• ReCoM: Realistic Co-Speech Motion Generation with Recurrent Embedded Transformer [58.49950218437718]
  We present ReCoM, an efficient framework for generating high-fidelity and generalizable human body motions synchronized with speech. The core innovation lies in the Recurrent Embedded Transformer (RET), which integrates Dynamic Embedding Regularization (DER) into a Vision Transformer (ViT) core architecture. To enhance model robustness, we incorporate the proposed DER strategy, which equips the model with dual capabilities of noise resistance and cross-domain generalization.
  arXiv  Detail & Related papers  (2025-03-27T16:39:40Z)
• Ready-to-React: Online Reaction Policy for Two-Character Interaction Generation [82.73098356401725]
  We propose an online reaction policy, called Ready-to-React, to generate the next character pose based on past observed motions.<n>Each character has its own reaction policy as its "brain", enabling them to interact like real humans in a streaming manner.<n>Our approach can be controlled by sparse signals, making it well-suited for VR and other online interactive environments.
  arXiv  Detail & Related papers  (2025-02-27T18:40:30Z)
• Two-in-One: Unified Multi-Person Interactive Motion Generation by Latent Diffusion Transformer [24.166147954731652]
  Multi-person interactive motion generation is a critical yet under-explored domain in computer character animation.<n>Current research often employs separate module branches for individual motions, leading to a loss of interaction information.<n>We propose a novel, unified approach that models multi-person motions and their interactions within a single latent space.
  arXiv  Detail & Related papers  (2024-12-21T15:35:50Z)
• Fast and Robust Visuomotor Riemannian Flow Matching Policy [15.341017260123927]
  Diffusion-based visuomotor policies excel at learning complex robotic tasks.<n>RFMP is a model that inherits the easy training and fast inference capabilities of flow matching.
  arXiv  Detail & Related papers  (2024-12-14T15:03:33Z)
• ReGenNet: Towards Human Action-Reaction Synthesis [87.57721371471536]
  We analyze the asymmetric, dynamic, synchronous, and detailed nature of human-human interactions. We propose the first multi-setting human action-reaction benchmark to generate human reactions conditioned on given human actions.
  arXiv  Detail & Related papers  (2024-03-18T15:33:06Z)
• ReMoS: 3D Motion-Conditioned Reaction Synthesis for Two-Person Interactions [66.87211993793807]
  We present ReMoS, a denoising diffusion based model that synthesizes full body motion of a person in two person interaction scenario. We demonstrate ReMoS across challenging two person scenarios such as pair dancing, Ninjutsu, kickboxing, and acrobatics. We also contribute the ReMoCap dataset for two person interactions containing full body and finger motions.
  arXiv  Detail & Related papers  (2023-11-28T18:59:52Z)
• InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusion [29.25063155767897]
  This paper addresses a novel task of anticipating 3D human-object interactions (HOIs) Our task is significantly more challenging, as it requires modeling dynamic objects with various shapes, capturing whole-body motion, and ensuring physically valid interactions. Experiments on multiple human-object interaction datasets demonstrate the effectiveness of our method for this task, capable of producing realistic, vivid, and remarkably long-term 3D HOI predictions.
  arXiv  Detail & Related papers  (2023-08-31T17:59:08Z)
• Persistent-Transient Duality: A Multi-mechanism Approach for Modeling Human-Object Interaction [58.67761673662716]
  Humans are highly adaptable, swiftly switching between different modes to handle different tasks, situations and contexts. In Human-object interaction (HOI) activities, these modes can be attributed to two mechanisms: (1) the large-scale consistent plan for the whole activity and (2) the small-scale children interactive actions that start and end along the timeline. This work proposes to model two concurrent mechanisms that jointly control human motion.
  arXiv  Detail & Related papers  (2023-07-24T12:21:33Z)
• Interactive Character Control with Auto-Regressive Motion Diffusion Models [18.727066177880708]
  We propose A-MDM (Auto-regressive Motion Diffusion Model) for real-time motion synthesis. Our conditional diffusion model takes an initial pose as input, and auto-regressively generates successive motion frames conditioned on previous frame. We introduce a suite of techniques for incorporating interactive controls into A-MDM, such as task-oriented sampling, in-painting, and hierarchical reinforcement learning.
  arXiv  Detail & Related papers  (2023-06-01T07:48:34Z)
• BoDiffusion: Diffusing Sparse Observations for Full-Body Human Motion Synthesis [14.331548412833513]
  Mixed reality applications require tracking the user's full-body motion to enable an immersive experience. We propose BoDiffusion -- a generative diffusion model for motion synthesis to tackle this under-constrained reconstruction problem. We present a time and space conditioning scheme that allows BoDiffusion to leverage sparse tracking inputs while generating smooth and realistic full-body motion sequences.
  arXiv  Detail & Related papers  (2023-04-21T16:39:05Z)
• Controllable Motion Synthesis and Reconstruction with Autoregressive Diffusion Models [18.50942770933098]
  MoDiff is an autoregressive probabilistic diffusion model over motion sequences conditioned on control contexts of other modalities. Our model integrates a cross-modal Transformer encoder and a Transformer-based decoder, which are found effective in capturing temporal correlations in motion and control modalities.
  arXiv  Detail & Related papers  (2023-04-03T08:17:08Z)
• Interaction Transformer for Human Reaction Generation [61.22481606720487]
  We propose a novel interaction Transformer (InterFormer) consisting of a Transformer network with both temporal and spatial attentions. Our method is general and can be used to generate more complex and long-term interactions.
  arXiv  Detail & Related papers  (2022-07-04T19:30:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.