FreeScale: Scaling 3D Scenes via Certainty-Aware Free-View Generation

• URL: http://arxiv.org/abs/2604.10512v1
• Date: Sun, 12 Apr 2026 08:00:53 GMT
• Title: FreeScale: Scaling 3D Scenes via Certainty-Aware Free-View Generation
• Abstract summary: FreeScale is a framework that transforms limited real-world image sequences into a scalable source of high-quality training data.<n>We show FreeScale's effectiveness by scaling up the training of feedforward NVS models, achieving a notable gain of 2.7 dB in PSNR.<n>Our work provides a practical and powerful data generation engine to overcome a fundamental bottleneck in 3D vision.
• Score: 75.74617373156902
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: The development of generalizable Novel View Synthesis (NVS) models is critically limited by the scarcity of large-scale training data featuring diverse and precise camera trajectories. While real-world captures are photorealistic, they are typically sparse and discrete. Conversely, synthetic data scales but suffers from a domain gap and often lacks realistic semantics. We introduce FreeScale, a novel framework that leverages the power of scene reconstruction to transform limited real-world image sequences into a scalable source of high-quality training data. Our key insight is that an imperfect reconstructed scene serves as a rich geometric proxy, but naively sampling from it amplifies artifacts. To this end, we propose a certainty-aware free-view sampling strategy identifying novel viewpoints that are both semantically meaningful and minimally affected by reconstruction errors. We demonstrate FreeScale's effectiveness by scaling up the training of feedforward NVS models, achieving a notable gain of 2.7 dB in PSNR on challenging out-of-distribution benchmarks. Furthermore, we show that the generated data can actively enhance per-scene 3D Gaussian Splatting optimization, leading to consistent improvements across multiple datasets. Our work provides a practical and powerful data generation engine to overcome a fundamental bottleneck in 3D vision. Project page: https://mvp-ai-lab.github.io/FreeScale.

Related papers

• RayDer: Scalable Self-Supervised Novel View Synthesis from Real-World Video [24.21520175195074]
  RayDer is a feed-forward transformer that consolidates camera estimation, scene reconstruction, and rendering into a single backbone.<n>A minimal dynamic state, treated as a nuisance factor, absorbs time-varying content and enables stable training on real-world video.<n>RayDer exhibits clean power-law scaling with data and compute, and outperforms static-scene data mixtures.
  arXiv  Detail & Related papers  (2026-05-29T16:50:27Z)
• Scal3R: Scalable Test-Time Training for Large-Scale 3D Reconstruction [50.5449251266956]
  This paper addresses the task of large-scale 3D scene reconstruction from long video sequences.<n>Recent feed-forward reconstruction models have shown promising results by directly regressing 3D geometry from RGB images without explicit 3D priors or geometric constraints.<n>We propose a novel neural global context representation that efficiently compresses and retains long-range scene information.
  arXiv  Detail & Related papers  (2026-04-09T17:59:50Z)
• OracleGS: Grounding Generative Priors for Sparse-View Gaussian Splatting [78.70702961852119]
  OracleGS reconciles generative completeness with regressive fidelity for sparse view Gaussian Splatting.<n>Our approach conditions the powerful generative prior on multi-view geometric evidence, filtering hallucinatory artifacts while preserving plausible completions in under-constrained regions.
  arXiv  Detail & Related papers  (2025-09-27T11:19:32Z)
• MegaScenes: Scene-Level View Synthesis at Scale [69.21293001231993]
  Scene-level novel view synthesis (NVS) is fundamental to many vision and graphics applications. We create a large-scale scene-level dataset from Internet photo collections, called MegaScenes, which contains over 100K structure from motion (SfM) reconstructions from around the world. We analyze failure cases of state-of-the-art NVS methods and significantly improve generation consistency.
  arXiv  Detail & Related papers  (2024-06-17T17:55:55Z)
• Real3D: Scaling Up Large Reconstruction Models with Real-World Images [34.735198125706326]
  Real3D is the first LRM system that can be trained using single-view real-world images. We propose two unsupervised losses that allow us to supervise LRMs at the pixel- and semantic-level. We develop an automatic data curation approach to collect high-quality examples from in-the-wild images.
  arXiv  Detail & Related papers  (2024-06-12T17:59:08Z)
• Denoising Diffusion via Image-Based Rendering [54.20828696348574]
  We introduce the first diffusion model able to perform fast, detailed reconstruction and generation of real-world 3D scenes. First, we introduce a new neural scene representation, IB-planes, that can efficiently and accurately represent large 3D scenes. Second, we propose a denoising-diffusion framework to learn a prior over this novel 3D scene representation, using only 2D images.
  arXiv  Detail & Related papers  (2024-02-05T19:00:45Z)
• RandomRooms: Unsupervised Pre-training from Synthetic Shapes and Randomized Layouts for 3D Object Detection [138.2892824662943]
  A promising solution is to make better use of the synthetic dataset, which consists of CAD object models, to boost the learning on real datasets. Recent work on 3D pre-training exhibits failure when transfer features learned on synthetic objects to other real-world applications. In this work, we put forward a new method called RandomRooms to accomplish this objective.
  arXiv  Detail & Related papers  (2021-08-17T17:56:12Z)
• Learning to Drop Points for LiDAR Scan Synthesis [5.132259673802809]
  Generative modeling of 3D scenes is a crucial topic for aiding mobile robots to improve unreliable observations. Most existing studies on point clouds have focused on small and uniform-density data. 3D LiDAR point clouds widely used in mobile robots are non-trivial to be handled because of the large number of points and varying-density. This paper proposes a novel framework based on generative adversarial networks to synthesize realistic LiDAR data as an improved 2D representation.
  arXiv  Detail & Related papers  (2021-02-23T21:53:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.