PhyScene3D: Physically Consistent Interactive 3D Tabletop Scene Generation

• URL: http://arxiv.org/abs/2606.01649v2
• Date: Wed, 03 Jun 2026 08:26:24 GMT
• Title: PhyScene3D: Physically Consistent Interactive 3D Tabletop Scene Generation
• Abstract summary: Generating 3D tabletop scenes is a fundamental yet underexplored problem for interactive and generalist robotic learning.<n>Here, we introduce PhyScene3D, a framework that reformulates generation as a Human-Mimetic Constructive Process.<n>Experiments demonstrate that PhyScene3D outperforms state-of-the-art approaches in both semantic accuracy and physical validity.
• Score: 75.9028457575434
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generating physically consistent 3D tabletop scenes is a fundamental yet underexplored problem for interactive and generalist robotic learning. The challenge stems from dense object hierarchies and irregular affordances. Here, an interactive scene denotes a physically valid, collision-free environment directly loadable into physics simulators. Existing methods, ranging from decoupled symbolic solvers to end-to-end regression models, often suffer from error propagation or overfitting to noisy supervision containing widespread physical violations. To address these limitations, we introduce PhyScene3D, a framework that reformulates generation as a Human-Mimetic Constructive Process. The proposed Cognitive Topological Reasoning Chain (CTRC) factorizes scene synthesis into a sequential, anchor-conditioned process. It employs a 3D AABB-based placement scheme that imposes a strong structural inductive bias. To address imperfect supervision and physical infeasibility, we introduce Physics-Aware Denoising Alignment (PADA). It integrates a differentiable Signed Distance Field (SDF) with Test-Time Optimization (TTO) to project generated scenes onto a physics-feasible manifold while preserving semantic intent. Experiments demonstrate that PhyScene3D outperforms state-of-the-art approaches in both semantic accuracy and physical validity, achieving a 40% reduction in scene-wise collision rate relative to the human-annotated training data.

Related papers

• DeSeG: Decoupling Semantic Intent and Geometric Constraints for Physically Plausible Human-Scene Interaction [50.06148080975928]
  We propose DeSeG, a hierarchical framework that explicitly decouples semantic intent from geometric constraints.<n>DeSeG achieves state-of-the-art performance, reducing mean scene penetration by 47% and improving semantic alignment by 29% over the SOTA baselines.
  arXiv  Detail & Related papers  (2026-07-07T03:21:08Z)
• REST3D: Reconstructing Physically Stable 3D Scenes from a Single Image [31.061246129846044]
  Reconstructing physically stable 3D scenes from a single RGB image enables casual images to be converted into simulation-ready digital assets.<n>We propose REST3D, a single-image reconstruction framework that can reconstruct physically stable 3D scenes by integrating physical scene understanding with physics-constrained refinement.
  arXiv  Detail & Related papers  (2026-05-28T17:59:01Z)
• STABLE: Simulation-Ready Tabletop Layout Generation via a Semantics-Physics Dual System [76.3920413336862]
  We present STABLE, a semantics-physics dual-system tailored for simulation-ready tabletop scene generation.<n>STABLE consists of two complementary modules: a Semantic Reasoner and a Physics Corrector.<n> Experiments demonstrate that STABLE successfully generates simulation-ready tabletop scenes that strictly conform to task instructions.
  arXiv  Detail & Related papers  (2026-05-15T16:18:42Z)
• SceneTeract: Agentic Functional Affordances and VLM Grounding in 3D Scenes [49.117927512531715]
  We introduce SceneTeract, a framework that verifies 3D scene functionality under agent-specific constraints.<n> SceneTeract decomposes complex activities into sequences of atomic actions and validates each step against accessibility requirements.<n>We release the SceneTeract verification suite and data to bridge perception and physical reality in embodied 3D scene understanding.
  arXiv  Detail & Related papers  (2026-03-31T14:31:18Z)
• Generation Models Know Space: Unleashing Implicit 3D Priors for Scene Understanding [50.098085774845195]
  We propose a paradigm shift by leveraging the implicit spatial prior within large-scale generation models.<n>We introduce GeneEGA-3D (Video Extracted Generative Awareness), a plug-and-play framework that repurposes a pre-trained video diffusion model as a Latent World Simulator.
  arXiv  Detail & Related papers  (2026-03-19T17:59:58Z)
• Motion Forcing: A Decoupled Framework for Robust Video Generation in Motion Dynamics [37.22501359080204]
  We introduce textbfMotion Forcing, a framework designed to stabilize a video generation equilibrium.<n>Our key insight is to explicitly decouple physical reasoning from visual synthesis.<n>Experiments on autonomous driving benchmarks show that Motion Forcing significantly outperforms state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-03-11T04:44:46Z)
• Causal Reasoning Elicits Controllable 3D Scene Generation [35.22855710229319]
  CausalStruct is a novel framework that embeds causal reasoning into 3D scene generation.<n>We construct causal graphs where nodes represent objects and attributes, while edges encode causal dependencies and physical constraints.<n>Our method uses text or images to guide object placement and layout in 3D scenes, with 3D Gaussian Splatting and Score Distillation Sampling improving shape accuracy and rendering stability.
  arXiv  Detail & Related papers  (2025-09-18T01:03:21Z)
• Mixed Diffusion for 3D Indoor Scene Synthesis [55.94569112629208]
  We present MiDiffusion, a novel mixed discrete-continuous diffusion model designed to synthesize plausible 3D indoor scenes.<n>We show it outperforms state-of-the-art autoregressive and diffusion models in floor-conditioned 3D scene synthesis.
  arXiv  Detail & Related papers  (2024-05-31T17:54:52Z)
• gradSim: Differentiable simulation for system identification and visuomotor control [66.37288629125996]
  We present gradSim, a framework that overcomes the dependence on 3D supervision by leveraging differentiable multiphysics simulation and differentiable rendering. Our unified graph enables learning in challenging visuomotor control tasks, without relying on state-based (3D) supervision.
  arXiv  Detail & Related papers  (2021-04-06T16:32:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.