Scene-Level Heterogeneous Physics Simulation with 3D Gaussian Splats

• URL: http://arxiv.org/abs/2606.21753v1
• Date: Fri, 19 Jun 2026 21:12:52 GMT
• Title: Scene-Level Heterogeneous Physics Simulation with 3D Gaussian Splats
• Abstract summary: 3D Gaussian Splatting (3DGS) has achieved state-of-the-art rendering, but the representation gap prevents these assets from being physically interactive.<n>We propose a novel framework that bridges this gap by enabling 3DGS assets to participate in scene-level, heterogeneous, multi-solver physical simulations.<n>We demonstrate complex, two-way interactions between deformable 3DGS assets, standard CG assets such as fluids and meshes, and large-scale captured static environments.
• Score: 32.12749945486662
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: 3D Gaussian Splatting (3DGS) has achieved state-of-the-art photorealistic rendering, but the representation gap prevents these assets from being physically interactive. Production-grade physics engines do not understand the 3DGS representation, while prior physics-for-3DGS methods are monolithic silos. These prior works are fundamentally limited, demonstrating only object-centric physics in isolated environments, such as on an ideal plane. They are incapable of interacting with complex static collision geometry or heterogeneous assets. We propose a novel framework that, for the first time, bridges this gap by enabling 3DGS assets to participate in scene-level, heterogeneous, multi-solver physical simulations. Our core contribution is a Representation Abstraction Framework that translates all diverse assets, including 3DGS, virtual meshes, and fluids, into a unified physical particle set. This abstraction is key to enabling complex behaviors, such as the non-rigid deformation of 3DGS assets, within a unified physics pipeline. This particle set, along with the static scene collision boundaries derived from scene capture, is processed within a solver-agnostic physics kernel. The physical results are then mapped back to drive each asset's specific visual reconstruction. This architecture unlocks capabilities impossible with prior art. We demonstrate complex, two-way interactions between deformable 3DGS assets, standard CG assets such as fluids and meshes, and large-scale captured static environments, showcasing realistic coupled phenomena that were previously unattainable.

Related papers

• Neural Voxel Dynamics: Learning Implicit 3D Physics via Volumetric Feature Advection [0.14323566945483496]
  We present a self-dimensional framework for learning implicit 3D physical dynamics directly from video-derived supervisory signals.<n>We address this by shifting the predictive bottleneck from 2D image space to a lifted' 3D Volumetric Latent Space.<n>Unlike state-of-the-art hybrid models that rely on explicit classical simulators for training and/or inference, our architecture tracks material states implicitly within high-supervised V-JEPA features.
  arXiv  Detail & Related papers  (2026-06-24T22:06:35Z)
• PhysX-Omni: Unified Simulation-Ready Physical 3D Generation for Rigid, Deformable, and Articulated Objects [60.069451811298315]
  We introduce PhysX- Omni, a unified framework for simulation-ready physical 3D generation across diverse asset types.<n>Specifically, we develop a novel and efficient geometry representation tailored for Vision-Language Models, which directly encodes high-resolution 3D structures without compression.<n>In addition, we construct the first general simulation-ready 3D dataset, PhysXVerse, covering diverse indoor and outdoor categories.
  arXiv  Detail & Related papers  (2026-05-20T17:59:01Z)
• TelePhysics: Physics-Grounded Multi-Object Scene Generation from a Single Image with Real-Time Interaction [51.01447538245441]
  We propose TelePhysics, a training-free framework that converts a single image into a physically consistent and controllable video.<n>By representing the full scene geometry in a unified spatial coordinate system, TelePhysics resolves object penetration and alignment ambiguity.<n> Experimental results demonstrate that TelePhysics substantially outperforms prior methods in physical fidelity, spatial coherence, and controllability.
  arXiv  Detail & Related papers  (2026-05-19T08:16:44Z)
• PG-3DGS: Optimizing 3D Gaussian Splatting to Satisfy Physics Objectives [14.59375357686527]
  Physics-Guided 3D Gaussian Splatting (PG-3DGS) is a framework that couples differentiable physics simulation with 3D Gaussian representations to generate 3D structures.<n>Our approach produces geometries that are not only photometrically accurate but also physically functional.<n>Experiments on pouring and aerodynamic lift tasks show that PG-3DGS improves physical functionality while preserving visual quality.
  arXiv  Detail & Related papers  (2026-05-11T21:43:43Z)
• AniGen: Unified $S^3$ Fields for Animatable 3D Asset Generation [64.1405325883561]
  We present AniGen, a unified framework that directly generates animate-ready 3D assets conditioned on a single image.<n>Our key insight is to represent shape, skeleton, and skinning as mutually consistent $S3$ Fields (Shape, Skeleton, Skin) defined over a shared spatial domain.<n>AniGen substantially outperforms state-of-the-art sequential baselines in rig validity and animation quality.
  arXiv  Detail & Related papers  (2026-04-09T20:22:06Z)
• MotionAnymesh: Physics-Grounded Articulation for Simulation-Ready Digital Twins [25.25391483526521]
  MotionAnymesh is a framework that seamlessly transforms unstructured static meshes into simulation-ready digital twins.<n>Our method features a kinematic-aware part segmentation module that grounds VLM reasoning with explicit SP4D physical priors.
  arXiv  Detail & Related papers  (2026-03-13T12:30:42Z)
• PhysX-Anything: Simulation-Ready Physical 3D Assets from Single Image [67.76547268461411]
  PhysX-Anything is the first simulation-ready physical 3D generative framework.<n>It produces high-quality sim-ready 3D assets with explicit geometry, articulation, and physical attributes.<n>It reduces the number of tokens by 193x, enabling explicit geometry learning within standard VLM token budgets.
  arXiv  Detail & Related papers  (2025-11-17T17:59:53Z)
• Seed3D 1.0: From Images to High-Fidelity Simulation-Ready 3D Assets [63.67760219308476]
  We present Seed3D 1.0, a foundation model that generates simulation-ready 3D assets from single images.<n>Unlike existing 3D generation models, our system produces assets with accurate geometry, well-aligned textures, and realistic physically-based materials.
  arXiv  Detail & Related papers  (2025-10-22T18:16:32Z)
• PIG: Physically-based Multi-Material Interaction with 3D Gaussians [14.097146027458368]
  PIG: Physically-Based Multi-Material Interaction with 3D Gaussians is a novel approach that combines 3D object segmentation with the simulation of objects interacting in high precision.<n>We show that our method not only outperforms the state-of-the-art (SOTA) in terms of visual quality, but also opens up new directions and pipelines for the field of physically realistic scene generation.
  arXiv  Detail & Related papers  (2025-06-09T11:25:21Z)
• DecoupledGaussian: Object-Scene Decoupling for Physics-Based Interaction [21.80091691062415]
  We present DecoupledGaussian, a novel system that decouples static objects from their contacted surfaces captured in-the-wild videos.<n>We validate DecoupledGaussian through a comprehensive user study and quantitative benchmarks.<n>This system enhances digital interaction with objects and scenes in real-world environments, benefiting industries such as VR, robotics, and autonomous driving.
  arXiv  Detail & Related papers  (2025-03-07T14:54:54Z)
• Occlusion resistant learning of intuitive physics from videos [52.25308231683798]
  Key ability for artificial systems is to understand physical interactions between objects, and predict future outcomes of a situation. This ability, often referred to as intuitive physics, has recently received attention and several methods were proposed to learn these physical rules from video sequences.
  arXiv  Detail & Related papers  (2020-04-30T19:35:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.