Lucida: Parse, Generate, and Place for Composable Real-to-Sim Scene Modeling

• URL: http://arxiv.org/abs/2608.30821v1
• Date: Mon, 31 Aug 2026 14:03:20 GMT
• Title: Lucida: Parse, Generate, and Place for Composable Real-to-Sim Scene Modeling
• Abstract summary: Composable scene modeling aims to recover a real indoor scene as complete, editable object assets arranged as observed.<n>Lucia parses the video into a scene graph whose nodes carry per-instance multi-view evidence.<n>Lucia places assets with GizmoAct, a VLM policy that casts placement as multi-turn GUI interaction.
• Score: 11.305288161442995
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Composable scene modeling aims to recover a real indoor scene as complete, editable object assets arranged as observed, giving robot simulation and embodied AI a simulation-ready replica of the real environment whose objects can be manipulated individually. Existing pipelines decompose the task into three steps---parse the observations into instances, generate an asset for each, and place each asset back---but every step presumes an input that a cluttered capture rarely provides: accurate instance geometry, unoccluded views, and assets that accurately match the observations. We propose Lucida, which keeps this order but redistributes the requirements, so each step consumes only what a real capture reliably provides and precision is reached at the end of the pipeline rather than demanded at its start. Lucida parses the video into a scene graph whose nodes carry per-instance multi-view evidence, generates a complete asset for each instance from its evidence, and places assets with GizmoAct, a VLM policy that casts placement as multi-turn GUI interaction, manipulating the object's gizmo in a closed loop and deciding itself when alignment is reached. Across scene-level 3D object detection, object pose estimation, and scene reconstruction, Lucida improves mAP over Boxer by 69% on R2S-Scene, raises ADD-SB@0.05 from 57.8% to 83.4% on CA-1M, and increases scene F-Score from 0.794 for SAM3D to 0.924.

Related papers

• WorldSculpt: Generating Compositional Worlds from Grounded Videos [50.427204240556854]
  We study the problem of generating a compositional 3D representation of a cluttered scene containing hundreds of objects.<n>This task is challenging in densely cluttered scenes, where objects heavily occlude one another and each view reveals only a fraction of their geometry.<n>We show that complex scenes with hundreds of objects can instead be generated compositionally by adapting a strong single-object 3D generative prior to multi-view observations.
  arXiv  Detail & Related papers  (2026-09-04T17:59:35Z)
• Scene-SAM3D: Multi-View Scene Asset Generation Without Fine-Tuning [51.11861596631423]
  We introduce Scene-SAM3D, a training-free framework that extends SAM3D from single-view object generation to multi-view scene asset generation.<n>Scene-SAM3D selects a compact set of complementary views, reducing observation redundancy while providing additional evidence for regions occluded in individual views.<n>It performs step-efficient latent velocity fusion to integrate multi-view evidence and suppress cross-view conflicts in canonical space.
  arXiv  Detail & Related papers  (2026-07-18T12:51:17Z)
• Sparse2Act: Learning Action-Aligned Sparse 3D Representations for Cross-Domain Robot Manipulation [52.00134393320209]
  We introduce Sparse2Act, an observation-action alignment framework for pretraining sparse point-cloud encoders.<n>Masked sparse 3D tokens are trained to organize scene features around the workspace motion paired with the observation.<n>On the LIBERO-10 benchmark, our method achieves 86.9% average success after 500 fine-tuning steps.
  arXiv  Detail & Related papers  (2026-06-10T23:56:01Z)
• GRAIL: Generating Humanoid Loco-Manipulation from 3D Assets and Video Priors [113.71148915419246]
  We present GRAIL, a digital generation pipeline that composes 3D assets, simulator-ready scenes, and priors from video foundation models (VFMs) to synthesize interactions without rebuilding physical environments or teleoperating the robot.<n>GRAIL starts from fully specified 3D configurations in which object geometry, camera parameters, metric scale, environment depth, and a robot-proportioned character are known before video generation and reused during reconstruction.<n>We retarget the recovered motions to a humanoid robot and train complementary task-general morphology trackers.<n>GRAIL produces over 20,000 sequences spanning pick-up, object manipulation, sitting
  arXiv  Detail & Related papers  (2026-06-03T17:57:45Z)
• Asset Harvester: Extracting 3D Assets from Autonomous Driving Logs for Simulation [63.01022057888141]
  Asset Harvester is an image-to-3D model and end-to-end pipeline that converts sparse, in-thewild object observations from real driving logs into complete, simulation-ready assets.<n>SparseViewDiT is explicitly designed to address limited-angle views and other real-world data challenges.
  arXiv  Detail & Related papers  (2026-04-20T16:20:57Z)
• SceneSmith: Agentic Generation of Simulation-Ready Indoor Scenes [19.995619927680476]
  SceneSmith builds environments from architectural layout to natural furniture population.<n>SceneSmith generates more objects than prior methods, with 2% interobject collisions and 96% of objects remaining stable under physics simulation.<n>SceneSmith environments can be used in an end-to-end pipeline for automatic policy evaluation.
  arXiv  Detail & Related papers  (2026-02-09T19:56:04Z)
• ArtiWorld: LLM-Driven Articulation of 3D Objects in Scenes [43.19849355456126]
  ArtiWorld is a scene-aware pipeline that localizes candidate articulable objects from textual scene descriptions.<n>At the core of this pipeline is Arti4URDF, which leverages 3D point cloud and prior knowledge of a large language model.<n>We evaluate ArtiWorld at three levels: 3D simulated objects, full 3D simulated scenes, and real-world scan scenes.
  arXiv  Detail & Related papers  (2025-11-17T04:59:21Z)
• PickScan: Object discovery and reconstruction from handheld interactions [99.99566882133179]
  We develop an interaction-guided and class-agnostic method to reconstruct 3D representations of scenes. Our main contribution is a novel approach to detecting user-object interactions and extracting the masks of manipulated objects. Compared to Co-Fusion, the only comparable interaction-based and class-agnostic baseline, this corresponds to a reduction in chamfer distance of 73%.
  arXiv  Detail & Related papers  (2024-11-17T23:09:08Z)
• Multi-Model 3D Registration: Finding Multiple Moving Objects in Cluttered Point Clouds [23.923838486208524]
  We investigate a variation of the 3D registration problem, named multi-model 3D registration. In the multi-model registration problem, we are given two point clouds picturing a set of objects at different poses. We want to simultaneously reconstruct how all objects moved between the two point clouds.
  arXiv  Detail & Related papers  (2024-02-16T18:01:43Z)
• CabiNet: Scaling Neural Collision Detection for Object Rearrangement with Procedural Scene Generation [54.68738348071891]
  We first generate over 650K cluttered scenes - orders of magnitude more than prior work - in diverse everyday environments. We render synthetic partial point clouds from this data and use it to train our CabiNet model architecture. CabiNet is a collision model that accepts object and scene point clouds, captured from a single-view depth observation.
  arXiv  Detail & Related papers  (2023-04-18T21:09:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.