PhysHOI: Physics-Based Imitation of Dynamic Human-Object Interaction

• URL: http://arxiv.org/abs/2312.04393v1
• Date: Thu, 7 Dec 2023 16:06:31 GMT
• Title: PhysHOI: Physics-Based Imitation of Dynamic Human-Object Interaction
• Authors: Yinhuai Wang, Jing Lin, Ailing Zeng, Zhengyi Luo, Jian Zhang and Lei Zhang
• Abstract summary: We present PhysHOI, the first physics-based whole-body HOI imitation approach without task-specific reward designs. Except for the kinematic HOI representations of humans and objects, we introduce the contact graph to model the contact relations between body parts and objects explicitly. Based on the key designs, PhysHOI can imitate diverse HOI tasks simply yet effectively without prior knowledge.
• Score: 22.48933099236595
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Humans interact with objects all the time. Enabling a humanoid to learn human-object interaction (HOI) is a key step for future smart animation and intelligent robotics systems. However, recent progress in physics-based HOI requires carefully designed task-specific rewards, making the system unscalable and labor-intensive. This work focuses on dynamic HOI imitation: teaching humanoid dynamic interaction skills through imitating kinematic HOI demonstrations. It is quite challenging because of the complexity of the interaction between body parts and objects and the lack of dynamic HOI data. To handle the above issues, we present PhysHOI, the first physics-based whole-body HOI imitation approach without task-specific reward designs. Except for the kinematic HOI representations of humans and objects, we introduce the contact graph to model the contact relations between body parts and objects explicitly. A contact graph reward is also designed, which proved to be critical for precise HOI imitation. Based on the key designs, PhysHOI can imitate diverse HOI tasks simply yet effectively without prior knowledge. To make up for the lack of dynamic HOI scenarios in this area, we introduce the BallPlay dataset that contains eight whole-body basketball skills. We validate PhysHOI on diverse HOI tasks, including whole-body grasping and basketball skills.

Related papers

• PhysWorld: From Real Videos to World Models of Deformable Objects via Physics-Aware Demonstration Synthesis [52.905353023326306]
  We propose PhysWorld, a framework that synthesizes physically plausible and diverse demonstrations to learn efficient world models.<n>Experiments show that PhysWorld has competitive performance while enabling inference speeds 47 times faster than the recent state-of-the-art method, i.e., PhysTwin.
  arXiv  Detail & Related papers  (2025-10-24T13:25:39Z)
• PA-HOI: A Physics-Aware Human and Object Interaction Dataset [15.985215689392179]
  We introduce the PA-HOI Motion Capture dataset, which highlights the impact of objects' physical attributes on human motion dynamics.<n>The dataset comprises 562 motion sequences of human-object interactions, with each sequence performed by subjects of different genders.<n>This dataset stands out by significantly extending the scope of existing ones for understanding how the physical attributes of different objects influence human posture, speed, motion scale, and interacting strategies.
  arXiv  Detail & Related papers  (2025-08-08T10:36:23Z)
• Half-Physics: Enabling Kinematic 3D Human Model with Physical Interactions [88.01918532202716]
  We introduce a novel approach that embeds SMPL-X into a tangible entity capable of dynamic physical interactions with its surroundings.<n>Our approach maintains kinematic control over inherent SMPL-X poses while ensuring physically plausible interactions with scenes and objects.<n>Unlike reinforcement learning-based methods, which demand extensive and complex training, our half-physics method is learning-free and generalizes to any body shape and motion.
  arXiv  Detail & Related papers  (2025-07-31T17:58:33Z)
• PhysGaia: A Physics-Aware Dataset of Multi-Body Interactions for Dynamic Novel View Synthesis [62.283499219361595]
  PhysGaia is a physics-aware dataset specifically designed for Dynamic Novel View Synthesis (DyNVS)<n>Our dataset provides complex dynamic scenarios with rich interactions among multiple objects.<n>PhysGaia will significantly advance research in dynamic view synthesis, physics-based scene understanding, and deep learning models integrated with physical simulation.
  arXiv  Detail & Related papers  (2025-06-03T12:19:18Z)
• HUMOTO: A 4D Dataset of Mocap Human Object Interactions [27.573065832588554]
  Human Motions with Objects is a high-fidelity dataset of human-object interactions for motion generation, computer vision, and robotics applications. Humoto captures interactions with 63 precisely modeled objects and 72 articulated parts. Professional artists rigorously clean and verify each sequence, minimizing foot sliding and object penetrations.
  arXiv  Detail & Related papers  (2025-04-14T16:59:29Z)
• AvatarGO: Zero-shot 4D Human-Object Interaction Generation and Animation [60.5897687447003]
  AvatarGO is a novel framework designed to generate realistic 4D HOI scenes from textual inputs. Our framework not only generates coherent compositional motions, but also exhibits greater robustness in handling issues. As the first attempt to synthesize 4D avatars with object interactions, we hope AvatarGO could open new doors for human-centric 4D content creation.
  arXiv  Detail & Related papers  (2024-10-09T17:58:56Z)
• PhysDreamer: Physics-Based Interaction with 3D Objects via Video Generation [62.53760963292465]
  PhysDreamer is a physics-based approach that endows static 3D objects with interactive dynamics. We present our approach on diverse examples of elastic objects and evaluate the realism of the synthesized interactions through a user study.
  arXiv  Detail & Related papers  (2024-04-19T17:41:05Z)
• FORCE: Dataset and Method for Intuitive Physics Guided Human-object Interaction [39.810254311528354]
  We introduce the FORCE model, a kinematic approach for diverse, nuanced human-object interactions by modeling physical attributes. Our key insight is that human motion is dictated by the interrelation between the force exerted by the human and the perceived resistance. Experiments also demonstrate incorporating human force facilitates learning multi-class motion.
  arXiv  Detail & Related papers  (2024-03-17T14:52:05Z)
• Compositional 3D Human-Object Neural Animation [93.38239238988719]
  Human-object interactions (HOIs) are crucial for human-centric scene understanding applications such as human-centric visual generation, AR/VR, and robotics. In this paper, we address this challenge in HOI animation from a compositional perspective. We adopt neural human-object deformation to model and render HOI dynamics based on implicit neural representations.
  arXiv  Detail & Related papers  (2023-04-27T10:04:56Z)
• Task-Oriented Human-Object Interactions Generation with Implicit Neural Representations [61.659439423703155]
  TOHO: Task-Oriented Human-Object Interactions Generation with Implicit Neural Representations. Our method generates continuous motions that are parameterized only by the temporal coordinate. This work takes a step further toward general human-scene interaction simulation.
  arXiv  Detail & Related papers  (2023-03-23T09:31:56Z)
• Full-Body Articulated Human-Object Interaction [61.01135739641217]
  CHAIRS is a large-scale motion-captured f-AHOI dataset consisting of 16.2 hours of versatile interactions. CHAIRS provides 3D meshes of both humans and articulated objects during the entire interactive process. By learning the geometrical relationships in HOI, we devise the very first model that leverage human pose estimation.
  arXiv  Detail & Related papers  (2022-12-20T19:50:54Z)
• SPACE: A Simulator for Physical Interactions and Causal Learning in 3D Environments [2.105564340986074]
  We introduce SPACE: A Simulator for Physical Interactions and Causal Learning in 3D Environments. Inspired by daily object interactions, the SPACE dataset comprises videos depicting three types of physical events: containment, stability and contact. We show that the SPACE dataset improves the learning of intuitive physics with an approach inspired by curriculum learning.
  arXiv  Detail & Related papers  (2021-08-13T11:49:46Z)
• Kinematics-Guided Reinforcement Learning for Object-Aware 3D Ego-Pose Estimation [25.03715978502528]
  We propose a method for incorporating object interaction and human body dynamics into the task of 3D ego-pose estimation. We use a kinematics model of the human body to represent the entire range of human motion, and a dynamics model of the body to interact with objects inside a physics simulator. This is the first work to estimate a physically valid 3D full-body interaction sequence with objects from egocentric videos.
  arXiv  Detail & Related papers  (2020-11-10T00:06:43Z)
• ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulation [75.0278287071591]
  ThreeDWorld (TDW) is a platform for interactive multi-modal physical simulation. TDW enables simulation of high-fidelity sensory data and physical interactions between mobile agents and objects in rich 3D environments. We present initial experiments enabled by TDW in emerging research directions in computer vision, machine learning, and cognitive science.
  arXiv  Detail & Related papers  (2020-07-09T17:33:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.