Synthesizing Physically Plausible Human Motions in 3D Scenes

• URL: http://arxiv.org/abs/2308.09036v2
• Date: Sun, 02 Mar 2025 17:15:30 GMT
• Title: Synthesizing Physically Plausible Human Motions in 3D Scenes
• Authors: Liang Pan, Jingbo Wang, Buzhen Huang, Junyu Zhang, Haofan Wang, Xu Tang, Yangang Wang,
• Abstract summary: We present a physics-based character control framework for synthesizing human-scene interactions.<n>Key idea is to decouple human-scene interactions into two fundamental processes, Interacting and Navigating.<n>To realize navigation in cluttered environments, we introduce NavCon, where a trajectory following policy enables characters to track pre-planned collision-free paths.
• Score: 39.16696410790455
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a physics-based character control framework for synthesizing human-scene interactions. Recent advances adopt physics simulation to mitigate artifacts produced by data-driven kinematic approaches. However, existing physics-based methods mainly focus on single-object environments, resulting in limited applicability in realistic 3D scenes with multi-objects. To address such challenges, we propose a framework that enables physically simulated characters to perform long-term interaction tasks in diverse, cluttered, and unseen 3D scenes. The key idea is to decouple human-scene interactions into two fundamental processes, Interacting and Navigating, which motivates us to construct two reusable Controllers, namely InterCon and NavCon. Specifically, InterCon uses two complementary policies to enable characters to enter or leave the interacting state with a particular object (e.g., sitting on a chair or getting up). To realize navigation in cluttered environments, we introduce NavCon, where a trajectory following policy enables characters to track pre-planned collision-free paths. Benefiting from the divide and conquer strategy, we can train all policies in simple environments and directly apply them in complex multi-object scenes through coordination from a rule-based scheduler. Video and code are available at https://github.com/liangpan99/InterScene.

Related papers

• HOSIG: Full-Body Human-Object-Scene Interaction Generation with Hierarchical Scene Perception [57.37135310143126]
  HO SIG is a novel framework for synthesizing full-body interactions through hierarchical scene perception.<n>Our framework supports unlimited motion length through autoregressive generation and requires minimal manual intervention.<n>This work bridges the critical gap between scene-aware navigation and dexterous object manipulation.
  arXiv  Detail & Related papers  (2025-06-02T12:08:08Z)
• Human-Object Interaction with Vision-Language Model Guided Relative Movement Dynamics [30.43930233035367]
  This paper introduces a unified Human-Object Interaction framework. It provides unified control over interactions with static scenes and dynamic objects using language commands. Our framework supports long-horizon interactions among dynamic, articulated, and static objects.
  arXiv  Detail & Related papers  (2025-03-24T05:18:04Z)
• MIMO: Controllable Character Video Synthesis with Spatial Decomposed Modeling [21.1274747033854]
  Character video synthesis aims to produce realistic videos of animatable characters within lifelike scenes. Milo is a novel framework which can synthesize character videos with controllable attributes. Milo achieves advanced scalability to arbitrary characters, generality to novel 3D motions, and applicability to interactive real-world scenes.
  arXiv  Detail & Related papers  (2024-09-24T15:00:07Z)
• PlaMo: Plan and Move in Rich 3D Physical Environments [68.75982381673869]
  We present PlaMo, a scene-aware path planner and a robust physics-based controller. The planner produces a sequence of motion paths, considering the various limitations the scene imposes on the motion. Our control policy generates rich and realistic physical motion adhering to the plan.
  arXiv  Detail & Related papers  (2024-06-26T10:41:07Z)
• Physics-based Scene Layout Generation from Human Motion [21.939444709132395]
  We present a physics-based approach that simultaneously optimize a scene layout generator and simulates a moving human in a physics simulator. We use reinforcement learning to perform a dual-optimization of both the character motion imitation controller and the scene layout generator. We evaluate our method using motions from SAMP and PROX, and demonstrate physically plausible scene layout reconstruction compared with the previous kinematics-based method.
  arXiv  Detail & Related papers  (2024-05-21T02:36:37Z)
• Generating Human Interaction Motions in Scenes with Text Control [66.74298145999909]
  We present TeSMo, a method for text-controlled scene-aware motion generation based on denoising diffusion models. Our approach begins with pre-training a scene-agnostic text-to-motion diffusion model. To facilitate training, we embed annotated navigation and interaction motions within scenes.
  arXiv  Detail & Related papers  (2024-04-16T16:04:38Z)
• Style-Consistent 3D Indoor Scene Synthesis with Decoupled Objects [84.45345829270626]
  Controllable 3D indoor scene synthesis stands at the forefront of technological progress. Current methods for scene stylization are limited to applying styles to the entire scene. We introduce a unique pipeline designed for synthesis 3D indoor scenes.
  arXiv  Detail & Related papers  (2024-01-24T03:10:36Z)
• Revisit Human-Scene Interaction via Space Occupancy [55.67657438543008]
  Human-scene Interaction (HSI) generation is a challenging task and crucial for various downstream tasks. In this work, we argue that interaction with a scene is essentially interacting with the space occupancy of the scene from an abstract physical perspective. By treating pure motion sequences as records of humans interacting with invisible scene occupancy, we can aggregate motion-only data into a large-scale paired human-occupancy interaction database.
  arXiv  Detail & Related papers  (2023-12-05T12:03:00Z)
• QuestEnvSim: Environment-Aware Simulated Motion Tracking from Sparse Sensors [69.75711933065378]
  We show that headset and controller pose can generate realistic full-body poses even in highly constrained environments. We discuss three features, the environment representation, the contact reward and scene randomization, crucial to the performance of the method.
  arXiv  Detail & Related papers  (2023-06-09T04:40:38Z)
• Synthesizing Diverse Human Motions in 3D Indoor Scenes [16.948649870341782]
  We present a novel method for populating 3D indoor scenes with virtual humans that can navigate in the environment and interact with objects in a realistic manner. Existing approaches rely on training sequences that contain captured human motions and the 3D scenes they interact with. We propose a reinforcement learning-based approach that enables virtual humans to navigate in 3D scenes and interact with objects realistically and autonomously.
  arXiv  Detail & Related papers  (2023-05-21T09:22:24Z)
• Synthesizing Physical Character-Scene Interactions [64.26035523518846]
  It is necessary to synthesize such interactions between virtual characters and their surroundings. We present a system that uses adversarial imitation learning and reinforcement learning to train physically-simulated characters. Our approach takes physics-based character motion generation a step closer to broad applicability.
  arXiv  Detail & Related papers  (2023-02-02T05:21:32Z)
• IMoS: Intent-Driven Full-Body Motion Synthesis for Human-Object Interactions [69.95820880360345]
  We present the first framework to synthesize the full-body motion of virtual human characters with 3D objects placed within their reach. Our system takes as input textual instructions specifying the objects and the associated intentions of the virtual characters. We show that our synthesized full-body motions appear more realistic to the participants in more than 80% of scenarios.
  arXiv  Detail & Related papers  (2022-12-14T23:59:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.