Seen2Scene: Completing Realistic 3D Scenes with Visibility-Guided Flow

• URL: http://arxiv.org/abs/2603.28548v1
• Date: Mon, 30 Mar 2026 15:08:38 GMT
• Title: Seen2Scene: Completing Realistic 3D Scenes with Visibility-Guided Flow
• Abstract summary: We present Seen2Scene, the first flow matching-based approach that trains on incomplete, real-world 3D scans for scene completion and generation.<n>By learning directly from real-world, incomplete 3D scans, Seen2Scene enables realistic 3D scene completion for complex, cluttered real environments.
• Score: 80.05357398241574
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present Seen2Scene, the first flow matching-based approach that trains directly on incomplete, real-world 3D scans for scene completion and generation. Unlike prior methods that rely on complete and hence synthetic 3D data, our approach introduces visibility-guided flow matching, which explicitly masks out unknown regions in real scans, enabling effective learning from real-world, partial observations. We represent 3D scenes using truncated signed distance field (TSDF) volumes encoded in sparse grids and employ a sparse transformer to efficiently model complex scene structures while masking unknown regions. We employ 3D layout boxes as an input conditioning signal, and our approach is flexibly adapted to various other inputs such as text or partial scans. By learning directly from real-world, incomplete 3D scans, Seen2Scene enables realistic 3D scene completion for complex, cluttered real environments. Experiments demonstrate that our model produces coherent, complete, and realistic 3D scenes, outperforming baselines in completion accuracy and generation quality.

Related papers

• Sparse auto-regressive modeling for scene generation from multi-view images [20.272184280044083]
  We introduce SPAR3S, a sparse voxel-aligned 3D latent generative model for conditional scene completion.<n>Our key insight is to formulate 3D scene generation in a structured, compact, voxel-aligned 3D latent space.<n>We demonstrate the effectiveness of our method on synthetic indoor scenes, achieving higher novel-view quality than prior work.
  arXiv  Detail & Related papers  (2026-09-03T14:41:34Z)
• MonoPlace3D: Learning 3D-Aware Object Placement for 3D Monocular Detection [24.611196145977022]
  Current monocular 3D detectors are held back by the limited diversity and scale of real-world datasets.<n>We show that where and how objects are positioned is just as crucial for training effective 3D monocular detectors.<n>We introduce MonoPlace3D, a novel system that considers the 3D scene content to create realistic augmentations.
  arXiv  Detail & Related papers  (2025-04-09T11:47:48Z)
• Enhancing Generalizability of Representation Learning for Data-Efficient 3D Scene Understanding [50.448520056844885]
  We propose a generative Bayesian network to produce diverse synthetic scenes with real-world patterns. A series of experiments robustly display our method's consistent superiority over existing state-of-the-art pre-training approaches.
  arXiv  Detail & Related papers  (2024-06-17T07:43:53Z)
• Volumetric Environment Representation for Vision-Language Navigation [66.04379819772764]
  Vision-language navigation (VLN) requires an agent to navigate through a 3D environment based on visual observations and natural language instructions. We introduce a Volumetric Environment Representation (VER), which voxelizes the physical world into structured 3D cells. VER predicts 3D occupancy, 3D room layout, and 3D bounding boxes jointly.
  arXiv  Detail & Related papers  (2024-03-21T06:14:46Z)
• BerfScene: Bev-conditioned Equivariant Radiance Fields for Infinite 3D Scene Generation [96.58789785954409]
  We propose a practical and efficient 3D representation that incorporates an equivariant radiance field with the guidance of a bird's-eye view map. We produce large-scale, even infinite-scale, 3D scenes via synthesizing local scenes and then stitching them with smooth consistency.
  arXiv  Detail & Related papers  (2023-12-04T18:56:10Z)
• Model2Scene: Learning 3D Scene Representation via Contrastive Language-CAD Models Pre-training [105.3421541518582]
  Current successful methods of 3D scene perception rely on the large-scale annotated point cloud. We propose Model2Scene, a novel paradigm that learns free 3D scene representation from Computer-Aided Design (CAD) models and languages. Model2Scene yields impressive label-free 3D object salient detection with an average mAP of 46.08% and 55.49% on the ScanNet and S3DIS datasets, respectively.
  arXiv  Detail & Related papers  (2023-09-29T03:51:26Z)
• Learnable Earth Parser: Discovering 3D Prototypes in Aerial Scans [20.030706182672144]
  We propose an unsupervised method for parsing large 3D scans of real-world scenes with easily-interpretable shapes. Our approach is based on a probabilistic reconstruction model that decomposes an input 3D point cloud into a small set of learned 3D shapes. We demonstrate the usefulness of our model on a novel dataset of seven large aerial LiDAR scans from diverse real-world scenarios.
  arXiv  Detail & Related papers  (2023-04-19T14:49:31Z)
• Learning 3D Scene Priors with 2D Supervision [37.79852635415233]
  We propose a new method to learn 3D scene priors of layout and shape without requiring any 3D ground truth. Our method represents a 3D scene as a latent vector, from which we can progressively decode to a sequence of objects characterized by their class categories. Experiments on 3D-FRONT and ScanNet show that our method outperforms state of the art in single-view reconstruction.
  arXiv  Detail & Related papers  (2022-11-25T15:03:32Z)
• CompNVS: Novel View Synthesis with Scene Completion [83.19663671794596]
  We propose a generative pipeline performing on a sparse grid-based neural scene representation to complete unobserved scene parts. We process encoded image features in 3D space with a geometry completion network and a subsequent texture inpainting network to extrapolate the missing area. Photorealistic image sequences can be finally obtained via consistency-relevant differentiable rendering.
  arXiv  Detail & Related papers  (2022-07-23T09:03:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.