Scene-SAM3D: Multi-View Scene Asset Generation Without Fine-Tuning

• URL: http://arxiv.org/abs/2607.16805v1
• Date: Sat, 18 Jul 2026 12:51:17 GMT
• Title: Scene-SAM3D: Multi-View Scene Asset Generation Without Fine-Tuning
• Abstract summary: We introduce Scene-SAM3D, a training-free framework that extends SAM3D from single-view object generation to multi-view scene asset generation.<n>Scene-SAM3D selects a compact set of complementary views, reducing observation redundancy while providing additional evidence for regions occluded in individual views.<n>It performs step-efficient latent velocity fusion to integrate multi-view evidence and suppress cross-view conflicts in canonical space.
• Score: 51.11861596631423
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: High-quality 3D scene assets are critical for embodied applications such as robotic manipulation, navigation, and simulation. Despite their strong object priors, recent single-image 3D generation models such as SAM3D remain insufficient for real-world scenes, where severe occlusions, redundant observations, and cross-view inconsistencies make reliable scene generation challenging. We introduce Scene-SAM3D, a training-free framework that extends SAM3D from single-view object generation to calibrated multi-view scene asset generation. Scene-SAM3D selects a compact set of complementary views, reducing observation redundancy while providing additional evidence for regions occluded in individual views. Based on the selected views, it performs step-efficient latent velocity fusion to integrate multi-view evidence and suppress cross-view conflicts in canonical space. Finally, a lightweight rigid-object Gaussian optimization refines the scene layout within 200 iterations while preserving the generated object geometry. Experiments on Replica and ScanNet++ demonstrate consistent improvements at both instance and scene levels, with our method reducing scene-level CD by 43.8% on Replica and 30.9% on ScanNet++, while cutting flow-model sampling FLOPs and wall-time latency by nearly 20% under the same multi-view setting. Code will be released at https://github.com/xibi777/Scene-SAM3D.

Related papers

• WorldSculpt: Generating Compositional Worlds from Grounded Videos [50.427204240556854]
  We study the problem of generating a compositional 3D representation of a cluttered scene containing hundreds of objects.<n>This task is challenging in densely cluttered scenes, where objects heavily occlude one another and each view reveals only a fraction of their geometry.<n>We show that complex scenes with hundreds of objects can instead be generated compositionally by adapting a strong single-object 3D generative prior to multi-view observations.
  arXiv  Detail & Related papers  (2026-09-04T17:59:35Z)
• Lucida: Parse, Generate, and Place for Composable Real-to-Sim Scene Modeling [11.305288161442995]
  Composable scene modeling aims to recover a real indoor scene as complete, editable object assets arranged as observed.<n>Lucia parses the video into a scene graph whose nodes carry per-instance multi-view evidence.<n>Lucia places assets with GizmoAct, a VLM policy that casts placement as multi-turn GUI interaction.
  arXiv  Detail & Related papers  (2026-08-31T14:03:20Z)
• ReScene: Structured Indoor Scene Reconstruction from Multi-View Captures [18.468596508491967]
  ReScene is a framework that threads multi-view geometry throughout the pipeline as a unifying prior.<n>Our method consists of two main components: HierView prioritizes reconstruction views based on semantic consistency and 3D coverage.<n>ReScene sets a new state of the art across geometry, rendering, and perceptual quality on a set of ScanNet scenes.
  arXiv  Detail & Related papers  (2026-06-26T13:08:40Z)
• SceneExpander: Expanding 3D Scenes with Free-Form Inserted Views [69.08965991211704]
  We study 3D scene expansion in a user-centric workflow, starting from a real scene captured by multi-view images.<n>Unlike simple object editing or style transfer in a fixed scene, the inserted view is often 3D-misaligned with the original reconstruction.<n>We propose SceneExpander, which applies test-time adaptation to a parametric feed-forward 3D reconstruction model.
  arXiv  Detail & Related papers  (2026-03-28T02:04:48Z)
• I3DM: Implicit 3D-aware Memory Retrieval and Injection for Consistent Video Scene Generation [56.33710337846449]
  I3DM is a novel implicit 3D-aware memory mechanism for consistent video scene generation.<n>At the core of our approach is a 3D-aware memory retrieval strategy.<n>To fully utilize the retrieved historical frames, we introduce a 3D-aligned memory injection module.
  arXiv  Detail & Related papers  (2026-03-24T16:45:40Z)
• MV-SAM3D: Adaptive Multi-View Fusion for Layout-Aware 3D Generation [19.245305728103748]
  We present MV-SAM3D, a training-free framework that extends layout-aware 3D generation with multi-view consistency and physical plausibility.<n>We propose two adaptive weighting strategies -- attention-entropy weighting and visibility weighting -- that enable confidence-aware fusion.<n>Experiments on standard benchmarks and real-world multi-object scenes demonstrate significant improvements in reconstruction fidelity and layout plausibility.
  arXiv  Detail & Related papers  (2026-03-12T07:53:35Z)
• Fake It To Make It: Virtual Multiviews to Enhance Monocular Indoor Semantic Scene Completion [0.8669877024051931]
  Monocular Indoor Semantic Scene Completion aims to reconstruct a 3D semantic occupancy map from a single RGB image of an indoor scene.<n>We introduce an innovative approach that leverages novel view synthesis and multiview fusion.<n>We demonstrate IoU score improvements of up to 2.8% for Scene Completion and 4.9% for Semantic Scene Completion when integrated with existing SSC networks on the NYUv2 dataset.
  arXiv  Detail & Related papers  (2025-03-07T02:09:38Z)
• MegaScenes: Scene-Level View Synthesis at Scale [69.21293001231993]
  Scene-level novel view synthesis (NVS) is fundamental to many vision and graphics applications. We create a large-scale scene-level dataset from Internet photo collections, called MegaScenes, which contains over 100K structure from motion (SfM) reconstructions from around the world. We analyze failure cases of state-of-the-art NVS methods and significantly improve generation consistency.
  arXiv  Detail & Related papers  (2024-06-17T17:55:55Z)
• Zero-Shot Multi-Object Scene Completion [59.325611678171974]
  We present a 3D scene completion method that recovers the complete geometry of multiple unseen objects in complex scenes from a single RGB-D image. Our method outperforms the current state-of-the-art on both synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2024-03-21T17:59:59Z)
• SAMPro3D: Locating SAM Prompts in 3D for Zero-Shot Instance Segmentation [24.733049281032272]
  We introduce SAMPro3D for zero-shot instance segmentation of 3D scenes.<n>Our approach segments 3D instances by applying the pretrained Segment Anything Model (SAM) to 2D frames.<n>Our method achieves comparable or better performance compared to previous zero-shot or fully supervised approaches.
  arXiv  Detail & Related papers  (2023-11-29T15:11:03Z)
• CabiNet: Scaling Neural Collision Detection for Object Rearrangement with Procedural Scene Generation [54.68738348071891]
  We first generate over 650K cluttered scenes - orders of magnitude more than prior work - in diverse everyday environments. We render synthetic partial point clouds from this data and use it to train our CabiNet model architecture. CabiNet is a collision model that accepts object and scene point clouds, captured from a single-view depth observation.
  arXiv  Detail & Related papers  (2023-04-18T21:09:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.