InterReal: A Unified Physics-Based Imitation Framework for Learning Human-Object Interaction Skills

• URL: http://arxiv.org/abs/2603.07516v1
• Date: Sun, 08 Mar 2026 07:44:35 GMT
• Title: InterReal: A Unified Physics-Based Imitation Framework for Learning Human-Object Interaction Skills
• Abstract summary: We develop a unified physics-based imitation learning framework for Real-world human-object Interaction (HOI) control.<n>InterReal enables humanoid robots to track HOI reference motions, facilitating the learning of fine-grained interactive skills.<n>A meta-policy guided by critical tracking error metrics explores and allocates reward signals to the low-level reinforcement learning objective.
• Score: 16.31202379413011
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Interaction is one of the core abilities of humanoid robots. However, most existing frameworks focus on non-interactive whole-body control, which limits their practical applicability. In this work, we develop InterReal, a unified physics-based imitation learning framework for Real-world human-object Interaction (HOI) control. InterReal enables humanoid robots to track HOI reference motions, facilitating the learning of fine-grained interactive skills and their deployment in real-world settings. Within this framework, we first introduce a HOI motion data augmentation scheme with hand-object contact constraints, and utilize the augmented motions to improve policy stability under object perturbations. Second, we propose an automatic reward learner to address the challenge of large-scale reward shaping. A meta-policy guided by critical tracking error metrics explores and allocates reward signals to the low-level reinforcement learning objective, which enables more effective learning of interactive policies. Experiments on HOI tasks of box-picking and box-pushing demonstrate that InterReal achieves the best tracking accuracy and the highest task success rate compared to recent baselines. Furthermore, we validate the framework on the real-world robot Unitree G1, which demonstrates its practical effectiveness and robustness beyond simulation.

Related papers

• VAIC: Vision-Guided Humanoid Agile Object Interaction Control via Decoupled Commands [56.029505206531155]
  Vision Guided Agile Interaction Control (VAIC) is a unified framework that bridges the gap between current controllers and humanoid robots.<n>First, a privileged teacher policy masters diverse interaction skills using precise object kinematics and exact environmental states.<n>Second, a deployable student policy distills these capabilities by replacing full body tracking with velocity targets across multiple axes.
  arXiv  Detail & Related papers  (2026-06-08T09:52:55Z)
• A Unified Multi-Layer Framework for Skill Acquisition from Imperfect Human Demonstrations [7.8260466406864575]
  Current Human-Robot Interaction (HRI) systems for skill teaching are fragmented.<n>Existing approaches in the literature do not offer a cohesive framework that is simultaneously efficient, intuitive, and universally safe.<n>This paper presents a novel, layered control framework that enables robust, compliant Learning from Demonstration.
  arXiv  Detail & Related papers  (2026-04-09T15:13:19Z)
• HAIC: Humanoid Agile Object Interaction Control via Dynamics-Aware World Model [56.4392302336014]
  We present HAIC, a framework for robust interaction across diverse object dynamics without external state estimation.<n>Our key contribution is a dynamics predictor that estimates high-order object states (velocity, acceleration) solely from proprioceptive history.<n> Experiments on a humanoid robot show HAIC achieves high success rates in agile tasks.
  arXiv  Detail & Related papers  (2026-02-12T09:34:35Z)
• Scalable Dexterous Robot Learning with AR-based Remote Human-Robot Interactions [8.111267700755986]
  This paper focuses on the scalable robot learning for manipulation in the dexterous robot arm-hand systems.<n>We present a unified framework to address the general manipulation task problem.
  arXiv  Detail & Related papers  (2026-02-07T03:47:21Z)
• InterPrior: Scaling Generative Control for Physics-Based Human-Object Interactions [58.329946838699044]
  Humans rarely plan whole-body interactions with objects at the level of explicit whole-body movements.<n>Scaling such priors is key to enabling humanoids to compose and generalize loco-manipulation skills.<n>We introduce InterPrior, a framework that learns a unified generative controller through large-scale imitation pretraining and post-training by reinforcement learning.
  arXiv  Detail & Related papers  (2026-02-05T18:59:27Z)
• OmniRetarget: Interaction-Preserving Data Generation for Humanoid Whole-Body Loco-Manipulation and Scene Interaction [76.44108003274955]
  A dominant paradigm for teaching humanoid robots complex skills is to retarget human motions as kinematic references to train reinforcement learning policies.<n>We introduce OmniRetarget, an interaction-preserving data generation engine based on an interaction mesh.<n>By minimizing the Laplacian deformation between the human and robot meshes, OmniRetarget generates kinematically feasible trajectories.
  arXiv  Detail & Related papers  (2025-09-30T17:59:02Z)
• Towards Immersive Human-X Interaction: A Real-Time Framework for Physically Plausible Motion Synthesis [51.95817740348585]
  Human-X is a novel framework designed to enable immersive and physically plausible human interactions across diverse entities.<n>Our method jointly predicts actions and reactions in real-time using an auto-regressive reaction diffusion planner.<n>Our framework is validated in real-world applications, including virtual reality interface for human-robot interaction.
  arXiv  Detail & Related papers  (2025-08-04T06:35:48Z)
• FLEX: A Framework for Learning Robot-Agnostic Force-based Skills Involving Sustained Contact Object Manipulation [9.292150395779332]
  We propose a novel framework for learning object-centric manipulation policies in force space.<n>Our method simplifies the action space, reduces unnecessary exploration, and decreases simulation overhead.<n>Our evaluations demonstrate that the method significantly outperforms baselines.
  arXiv  Detail & Related papers  (2025-03-17T17:49:47Z)
• InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactions [27.225777494300775]
  We introduce InterMimic, a framework that enables a single policy to robustly learn from hours of imperfect MoCap data.<n>Our experiments demonstrate that InterMimic produces realistic and diverse interactions across multiple HOI datasets.
  arXiv  Detail & Related papers  (2025-02-27T18:59:12Z)
• Learning Manipulation by Predicting Interaction [85.57297574510507]
  We propose a general pre-training pipeline that learns Manipulation by Predicting the Interaction. The experimental results demonstrate that MPI exhibits remarkable improvement by 10% to 64% compared with previous state-of-the-art in real-world robot platforms.
  arXiv  Detail & Related papers  (2024-06-01T13:28:31Z)
• RPMArt: Towards Robust Perception and Manipulation for Articulated Objects [56.73978941406907]
  We propose a framework towards Robust Perception and Manipulation for Articulated Objects ( RPMArt) RPMArt learns to estimate the articulation parameters and manipulate the articulation part from the noisy point cloud. We introduce an articulation-aware classification scheme to enhance its ability for sim-to-real transfer.
  arXiv  Detail & Related papers  (2024-03-24T05:55:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.