Learning to Assist: Physics-Grounded Human-Human Control via Multi-Agent Reinforcement Learning

• URL: http://arxiv.org/abs/2603.11346v1
• Date: Wed, 11 Mar 2026 22:25:44 GMT
• Title: Learning to Assist: Physics-Grounded Human-Human Control via Multi-Agent Reinforcement Learning
• Abstract summary: We formulate the imitation of closely interacting, force-exchanging human-human motion sequences as a multi-agent reinforcement learning problem.<n>We show that AssistMimic is the first method capable of successfully tracking assistive interaction motions on established benchmarks.
• Score: 29.898955971414154
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Humanoid robotics has strong potential to transform daily service and caregiving applications. Although recent advances in general motion tracking within physics engines (GMT) have enabled virtual characters and humanoid robots to reproduce a broad range of human motions, these behaviors are primarily limited to contact-less social interactions or isolated movements. Assistive scenarios, by contrast, require continuous awareness of a human partner and rapid adaptation to their evolving posture and dynamics. In this paper, we formulate the imitation of closely interacting, force-exchanging human-human motion sequences as a multi-agent reinforcement learning problem. We jointly train partner-aware policies for both the supporter (assistant) agent and the recipient agent in a physics simulator to track assistive motion references. To make this problem tractable, we introduce a partner policies initialization scheme that transfers priors from single-human motion-tracking controllers, greatly improving exploration. We further propose dynamic reference retargeting and contact-promoting reward, which adapt the assistant's reference motion to the recipient's real-time pose and encourage physically meaningful support. We show that AssistMimic is the first method capable of successfully tracking assistive interaction motions on established benchmarks, demonstrating the benefits of a multi-agent RL formulation for physically grounded and socially aware humanoid control.

Related papers

• Perceptive Behavior Foundation Model: Adapting Human Motion Priors to Robot-Centric Terrain [25.92545023668694]
  We develop a terrain-aware humanoid control framework that grounds human motion priors in robot-centric perception.<n>We train a blind adaptedreference teacher and transfer its terrain-conformal behavior to a deployed raw-reference student.<n>The student is an identity-gated Transformer tracker whose terrain features enter through residual pathways to preserve the motion-tracking prior and trained to produce local corrections only when needed.
  arXiv  Detail & Related papers  (2026-06-06T08:46:44Z)
• MeshMimic: Geometry-Aware Humanoid Motion Learning through 3D Scene Reconstruction [54.36564144414704]
  MeshMimic is an innovative framework that bridges 3D scene reconstruction and embodied intelligence to enable humanoid robots to learn coupled "motion-terrain" interactions directly from video.<n>By leveraging state-of-the-art 3D vision models, our framework precisely segments and reconstructs both human trajectories and the underlying 3D geometry of terrains and objects.
  arXiv  Detail & Related papers  (2026-02-17T17:09:45Z)
• TeleGate: Whole-Body Humanoid Teleoperation via Gated Expert Selection with Motion Prior [27.930225667015872]
  This paper presents TeleGate, a unified whole-body teleoperation framework for humanoid robots.<n>Our key idea is to preserve the full capability of domain-specific expert policies by training a lightweight gating network.<n>To compensate for the absence of future reference trajectories in real-time teleoperation, we introduce a VAE-based motion prior module.
  arXiv  Detail & Related papers  (2026-02-10T10:14:06Z)
• ResMimic: From General Motion Tracking to Humanoid Whole-body Loco-Manipulation via Residual Learning [59.64325421657381]
  Humanoid whole-body loco-manipulation promises transformative capabilities for daily service and warehouse tasks.<n>We introduce ResMimic, a two-stage residual learning framework for precise and expressive humanoid control from human motion data.<n>Results show substantial gains in task success, training efficiency, and robustness over strong baselines.
  arXiv  Detail & Related papers  (2025-10-06T17:47:02Z)
• OmniRetarget: Interaction-Preserving Data Generation for Humanoid Whole-Body Loco-Manipulation and Scene Interaction [76.44108003274955]
  A dominant paradigm for teaching humanoid robots complex skills is to retarget human motions as kinematic references to train reinforcement learning policies.<n>We introduce OmniRetarget, an interaction-preserving data generation engine based on an interaction mesh.<n>By minimizing the Laplacian deformation between the human and robot meshes, OmniRetarget generates kinematically feasible trajectories.
  arXiv  Detail & Related papers  (2025-09-30T17:59:02Z)
• HHI-Assist: A Dataset and Benchmark of Human-Human Interaction in Physical Assistance Scenario [63.77482302352545]
  HHI-Assist is a dataset comprising motion capture clips of human-human interactions in assistive tasks.<n>Our work has the potential to significantly enhance robotic assistance policies.
  arXiv  Detail & Related papers  (2025-09-12T09:38:17Z)
• Towards Immersive Human-X Interaction: A Real-Time Framework for Physically Plausible Motion Synthesis [51.95817740348585]
  Human-X is a novel framework designed to enable immersive and physically plausible human interactions across diverse entities.<n>Our method jointly predicts actions and reactions in real-time using an auto-regressive reaction diffusion planner.<n>Our framework is validated in real-world applications, including virtual reality interface for human-robot interaction.
  arXiv  Detail & Related papers  (2025-08-04T06:35:48Z)
• I-CTRL: Imitation to Control Humanoid Robots Through Constrained Reinforcement Learning [8.97654258232601]
  We develop a framework to control humanoid robots through bounded residual reinforcement learning (I-CTRL)<n>I-CTRL excels in motion imitation with simple and unique rewards that generalize across five robots.<n>Our framework introduces an automatic priority scheduler to manage large-scale motion datasets.
  arXiv  Detail & Related papers  (2024-05-14T16:12:27Z)
• Universal Humanoid Motion Representations for Physics-Based Control [71.46142106079292]
  We present a universal motion representation that encompasses a comprehensive range of motor skills for physics-based humanoid control. We first learn a motion imitator that can imitate all of human motion from a large, unstructured motion dataset. We then create our motion representation by distilling skills directly from the imitator.
  arXiv  Detail & Related papers  (2023-10-06T20:48:43Z)
• Residual Force Control for Agile Human Behavior Imitation and Extended Motion Synthesis [32.22704734791378]
  Reinforcement learning has shown great promise for realistic human behaviors by learning humanoid control policies from motion capture data. It is still very challenging to reproduce sophisticated human skills like ballet dance, or to stably imitate long-term human behaviors with complex transitions. We propose a novel approach, residual force control (RFC), that augments a humanoid control policy by adding external residual forces into the action space.
  arXiv  Detail & Related papers  (2020-06-12T17:56:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.