Gen2Physics: Grounding Generated 3D Meshes in Physics via Multi-View Material Decomposition

• URL: http://arxiv.org/abs/2608.23869v1
• Date: Mon, 24 Aug 2026 22:18:44 GMT
• Title: Gen2Physics: Grounding Generated 3D Meshes in Physics via Multi-View Material Decomposition
• Abstract summary: Gen2Physics grounds generated meshes in physics by automatically decomposing them into constituent material components.<n>Our pipeline integrates a fine-tuned Vision Transformer for dense material segmentation, a robust 2D-to-3D consistency projection, and a Vision-Language Model (VLM) guided refinement.
• Score: 10.183626550724352
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While state-of-the-art generative models produce high-fidelity 3D meshes, these outputs lack the physical properties required for interactive simulation, gaming, or robotics. We introduce Gen2Physics, a unified and automated framework that grounds generated meshes in physics by automatically decomposing them into their constituent material components. Unlike prior approaches, which focus on volumetric representations incompatible with standard physics engines, Gen2Physics operates directly on meshes to produce immediately simulation-ready assets. Our pipeline integrates a fine-tuned Vision Transformer for dense material segmentation, a robust 2D-to-3D consistency projection, and a Vision-Language Model (VLM) guided refinement that leverages contextual reasoning to assign physical properties and infer internal geometry (solid vs. hollow). By converting surface patches into volumes with distinct densities, our method enables physically plausible dynamic simulations. Experimental results on the ABO-500 and PartNet-Material benchmarks demonstrate that Gen2Physics more than doubles the material segmentation accuracy of prior physics-grounding pipelines (15.6 to 48.3 mIoU), while matching the mass-estimation accuracy of volumetric methods and being the only approach to output watertight per-material sub-meshes.

Related papers

• PhysiFormer: Learning to Simulate Mechanics in World Space [56.16644631495]
  We present PhysiFormer, a diffusion transformer for physically-plausible 3D object motion.<n>It represents objects as 3D meshes expressed in world coordinates.<n>It substantially outperforms autoregressive baselines in trajectory accuracy, rigidity preservation, and momentum-based physical consistency.
  arXiv  Detail & Related papers  (2026-06-25T17:59:00Z)
• PhysX-Omni: Unified Simulation-Ready Physical 3D Generation for Rigid, Deformable, and Articulated Objects [60.069451811298315]
  We introduce PhysX- Omni, a unified framework for simulation-ready physical 3D generation across diverse asset types.<n>Specifically, we develop a novel and efficient geometry representation tailored for Vision-Language Models, which directly encodes high-resolution 3D structures without compression.<n>In addition, we construct the first general simulation-ready 3D dataset, PhysXVerse, covering diverse indoor and outdoor categories.
  arXiv  Detail & Related papers  (2026-05-20T17:59:01Z)
• PhysForge: Generating Physics-Grounded 3D Assets for Interactive Virtual World [39.46909309051163]
  We introduce PhysForge, a framework for synthesizing physics-grounded 3D assets.<n>A "physical architect" acts as a "physical architect" to plan a "Hierarchical Physical Blueprint"<n>A physics-grounded diffusion model realizes this blueprint by high-fidelity geometry alongside precise kinematic parameters.
  arXiv  Detail & Related papers  (2026-05-06T17:33:23Z)
• PhyCo: Learning Controllable Physical Priors for Generative Motion [55.59209981836171]
  We present PhyCo, a framework that introduces continuous, interpretable, and physically grounded control into video generation.<n>Our approach integrates three key components: (i) a large-scale dataset of over 100K photorealistic simulation videos where friction, restitution, deformation, and force are systematically varied across diverse scenarios; (ii) physics-supervised fine-tuning of a pretrained diffusion model using a ControlNet conditioned on pixel-aligned physical property maps; and (iii) VLM-guided reward optimization, where a fine-tuned vision-language model evaluates generated videos with targeted physics queries and provides differentiable feedback.
  arXiv  Detail & Related papers  (2026-04-30T17:53:03Z)
• SLAT-Phys: Fast Material Property Field Prediction from Structured 3D Latents [55.26273954022457]
  Estimating the material property field of 3D assets is critical for physics-based simulation, robotics, and digital twin generation.<n>We present SLAT-Phys, an end-to-end method that predicts spatially varying material property fields of 3D assets directly from a single RGB image without explicit 3D reconstruction.
  arXiv  Detail & Related papers  (2026-03-25T06:14:03Z)
• MotionAnymesh: Physics-Grounded Articulation for Simulation-Ready Digital Twins [25.25391483526521]
  MotionAnymesh is a framework that seamlessly transforms unstructured static meshes into simulation-ready digital twins.<n>Our method features a kinematic-aware part segmentation module that grounds VLM reasoning with explicit SP4D physical priors.
  arXiv  Detail & Related papers  (2026-03-13T12:30:42Z)
• FastPhysGS: Accelerating Physics-based Dynamic 3DGS Simulation via Interior Completion and Adaptive Optimization [56.17833729527066]
  We propose FastPhysGS, a framework for physics-based dynamic 3DGS simulation.<n>FastPhysGS achieves high-fidelity physical simulation in 1 minute using only 7 GB runtime memory.
  arXiv  Detail & Related papers  (2026-02-02T07:00:42Z)
• PhysX-Anything: Simulation-Ready Physical 3D Assets from Single Image [67.76547268461411]
  PhysX-Anything is the first simulation-ready physical 3D generative framework.<n>It produces high-quality sim-ready 3D assets with explicit geometry, articulation, and physical attributes.<n>It reduces the number of tokens by 193x, enabling explicit geometry learning within standard VLM token budgets.
  arXiv  Detail & Related papers  (2025-11-17T17:59:53Z)
• GaussianProperty: Integrating Physical Properties to 3D Gaussians with LMMs [21.3615403516602]
  Estimating physical properties for visual data is a crucial task in computer vision, graphics, and robotics.<n>We introduce GaussianProperty, a training-free framework that assigns physical properties of materials to 3D Gaussians.<n>We demonstrate that 3D Gaussians with physical property annotations enable applications in physics-based dynamic simulation and robotic grasping.
  arXiv  Detail & Related papers  (2024-12-15T17:44:10Z)
• Phys4DGen: Physics-Compliant 4D Generation with Multi-Material Composition Perception [9.355276457984603]
  Phys4DGen is a novel 4D generation framework that integrates multi-material composition perception with physical simulation.<n>The framework achieves automated, physically plausible 4D generation through three innovative modules.<n> Experiments on both synthetic and real-world datasets demonstrate that Phys4DGen can generate high-fidelity 4D content with physical realism.
  arXiv  Detail & Related papers  (2024-11-25T12:12:38Z)
• PhysFlow: Unleashing the Potential of Multi-modal Foundation Models and Video Diffusion for 4D Dynamic Physical Scene Simulation [9.306758077479472]
  PhysFlow is a novel approach that leverages multi-modal foundation models and video diffusion to achieve enhanced 4D dynamic scene simulation.<n>This integrated framework enables accurate prediction and realistic simulation of dynamic interactions in real-world scenarios.
  arXiv  Detail & Related papers  (2024-11-21T18:55:23Z)
• DreamPhysics: Learning Physics-Based 3D Dynamics with Video Diffusion Priors [75.83647027123119]
  We propose to learn the physical properties of a material field with video diffusion priors.<n>We then utilize a physics-based Material-Point-Method simulator to generate 4D content with realistic motions.
  arXiv  Detail & Related papers  (2024-06-03T16:05:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.