PhysGraph: A Physics-aware 3D Scene Graph for Perception and Reasoning

• URL: http://arxiv.org/abs/2606.08655v1
• Date: Sun, 07 Jun 2026 14:50:44 GMT
• Title: PhysGraph: A Physics-aware 3D Scene Graph for Perception and Reasoning
• Abstract summary: We present PhysGraph, a framework that unifies symbolic reasoning with structured 3D geometry to model kinematic and physical properties in cluttered scenes.<n> PhysGraph achieves state-of-the-art results in semantic segmentation, multi-object mass estimation, and articulation prediction.
• Score: 24.361346589238035
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: To perform a wide range of daily tasks, robots need to construct a 3D representation that is semantically rich, physically grounded, and structured enough to support task planning and affordance prediction. However, existing approaches primarily focus on semantic retrieval, often overlooking physical and kinematic factors. Methods that attempt to model physical properties typically rely on narrow training sets or single-object modeling, limiting scalability and generalization across diverse object types. To address these challenges, we present PhysGraph, a framework that unifies symbolic reasoning with structured 3D geometry to model kinematic and physical properties in cluttered scenes. Given RGB-D observations, PhysGraph reconstructs object-centric 3D geometry and associates object instances across views. It then decomposes objects into functional parts and infers materials and articulations through visual reasoning. Evaluated on both synthetic and real-world datasets, PhysGraph achieves state-of-the-art results in semantic segmentation, multi-object mass estimation, and articulation prediction. With its simple yet effective design, PhysGraph produces physically consistent and semantically structured scene graphs, serving as a structured 3D representation for downstream tasks such as constraint-aware 3D affordance prediction and real-to-sim transfer, both of which are demonstrated in our experiments.

Related papers

• Learning Equivariant Neural-Augmented Object Dynamics From Few Interactions [9.357701422375824]
  A popular approach is to model objects as sets of 3D particles and learn their motion using graph neural networks.<n>We introduce PIEGraph, a novel approach to combining analytical physics and data-driven models.<n>We show that our method enables accurate dynamics prediction and reliable downstream robotic manipulation planning.
  arXiv  Detail & Related papers  (2026-05-04T15:11:22Z)
• ObjectForesight: Predicting Future 3D Object Trajectories from Human Videos [48.24897274501108]
  We introduce a 3D object-centric dynamics model that predicts future 6-DoF poses and trajectories of rigid objects from short egocentric video sequences.<n>Unlike conventional world or dynamics models that operate in pixel or latent space, ObjectForesight represents the world explicitly in 3D at the object level.<n>We show that ObjectForesight achieves significant gains in accuracy, geometric consistency, and generalization to unseen objects and scenes.
  arXiv  Detail & Related papers  (2026-01-08T18:58:08Z)
• TRACE: Learning 3D Gaussian Physical Dynamics from Multi-view Videos [7.616167860385134]
  We propose a new framework named TRACE to model the motion physics of complex dynamic 3D scenes.<n>By formulating each 3D point as a rigid particle with size and orientation in space, we directly learn a translation rotation dynamics system for each particle.
  arXiv  Detail & Related papers  (2025-08-13T13:43:01Z)
• PhysX-3D: Physical-Grounded 3D Asset Generation [48.78065667043986]
  Existing 3D generation primarily emphasizes geometries and textures while neglecting physical-grounded modeling.<n>We present PhysXNet - the first physics-grounded 3D dataset systematically annotated across five foundational dimensions.<n>We also propose textbfPhysXGen, a feed-forward framework for physics-grounded image-to-3D asset generation.
  arXiv  Detail & Related papers  (2025-07-16T17:59:35Z)
• SIGHT: Synthesizing Image-Text Conditioned and Geometry-Guided 3D Hand-Object Trajectories [124.24041272390954]
  Modeling hand-object interaction priors holds significant potential to advance robotic and embodied AI systems.<n>We introduce SIGHT, a novel task focused on generating realistic and physically plausible 3D hand-object interaction trajectories from a single image.<n>We propose SIGHT-Fusion, a novel diffusion-based image-text conditioned generative model that tackles this task by retrieving the most similar 3D object mesh from a database.
  arXiv  Detail & Related papers  (2025-03-28T20:53:20Z)
• Physically Compatible 3D Object Modeling from a Single Image [109.98124149566927]
  We present a framework that transforms single images into 3D physical objects.<n>Our framework embeds physical compatibility into the reconstruction process.<n>It consistently enhances the physical realism of 3D models over existing methods.
  arXiv  Detail & Related papers  (2024-05-30T21:59:29Z)
• SUGAR: Pre-training 3D Visual Representations for Robotics [85.55534363501131]
  We introduce a novel 3D pre-training framework for robotics named SUGAR. SUGAR captures semantic, geometric and affordance properties of objects through 3D point clouds. We show that SUGAR's 3D representation outperforms state-of-the-art 2D and 3D representations.
  arXiv  Detail & Related papers  (2024-04-01T21:23:03Z)
• 3D-IntPhys: Towards More Generalized 3D-grounded Visual Intuitive Physics under Challenging Scenes [68.66237114509264]
  We present a framework capable of learning 3D-grounded visual intuitive physics models from videos of complex scenes with fluids. We show our model can make long-horizon future predictions by learning from raw images and significantly outperforms models that do not employ an explicit 3D representation space.
  arXiv  Detail & Related papers  (2023-04-22T19:28:49Z)
• Occlusion resistant learning of intuitive physics from videos [52.25308231683798]
  Key ability for artificial systems is to understand physical interactions between objects, and predict future outcomes of a situation. This ability, often referred to as intuitive physics, has recently received attention and several methods were proposed to learn these physical rules from video sequences.
  arXiv  Detail & Related papers  (2020-04-30T19:35:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.