Horizon-GS: Unified 3D Gaussian Splatting for Large-Scale Aerial-to-Ground Scenes

• URL: http://arxiv.org/abs/2412.01745v1
• Date: Mon, 02 Dec 2024 17:42:00 GMT
• Title: Horizon-GS: Unified 3D Gaussian Splatting for Large-Scale Aerial-to-Ground Scenes
• Authors: Lihan Jiang, Kerui Ren, Mulin Yu, Linning Xu, Junting Dong, Tao Lu, Feng Zhao, Dahua Lin, Bo Dai,
• Abstract summary: We introduce Horizon-GS, a novel approach built upon Gaussian Splatting techniques, to tackle the unified reconstruction and rendering for aerial and street views. Our method addresses the key challenges of combining these perspectives with a new training strategy, overcoming viewpoint discrepancies to generate high-fidelity scenes.
• Score: 55.15494682493422
• License:
• Abstract: Seamless integration of both aerial and street view images remains a significant challenge in neural scene reconstruction and rendering. Existing methods predominantly focus on single domain, limiting their applications in immersive environments, which demand extensive free view exploration with large view changes both horizontally and vertically. We introduce Horizon-GS, a novel approach built upon Gaussian Splatting techniques, tackles the unified reconstruction and rendering for aerial and street views. Our method addresses the key challenges of combining these perspectives with a new training strategy, overcoming viewpoint discrepancies to generate high-fidelity scenes. We also curate a high-quality aerial-to-ground views dataset encompassing both synthetic and real-world scene to advance further research. Experiments across diverse urban scene datasets confirm the effectiveness of our method.

Related papers

• AerialGo: Walking-through City View Generation from Aerial Perspectives [48.53976414257845]
  AerialGo is a framework that generates realistic walking-through city views from aerial images. By conditioning ground-view synthesis on accessible aerial data, AerialGo bypasses the privacy risks inherent in ground-level imagery. Experiments show that AerialGo significantly enhances ground-level realism and structural coherence.
  arXiv  Detail & Related papers  (2024-11-29T08:14:07Z)
• Drone-assisted Road Gaussian Splatting with Cross-view Uncertainty [10.37108303188536]
  3D Gaussian Splatting (3D-GS) has made groundbreaking progress in neural rendering. The general fidelity of large-scale road scene renderings is often limited by the input imagery. We introduce the cross-view uncertainty to 3D-GS by matching the car-view ensemble-based rendering uncertainty to aerial images.
  arXiv  Detail & Related papers  (2024-08-27T17:59:55Z)
• SkyDiffusion: Ground-to-Aerial Image Synthesis with Diffusion Models and BEV Paradigm [14.492759165786364]
  Ground-to-aerial image synthesis focuses on generating realistic aerial images from corresponding ground street view images. We introduce SkyDiffusion, a novel cross-view generation method for synthesizing aerial images from street view images. We introduce a novel dataset, Ground2Aerial-3, designed for diverse ground-to-aerial image synthesis applications.
  arXiv  Detail & Related papers  (2024-08-03T15:43:56Z)
• Aerial Lifting: Neural Urban Semantic and Building Instance Lifting from Aerial Imagery [51.73680703579997]
  We present a neural radiance field method for urban-scale semantic and building-level instance segmentation from aerial images. objects in urban aerial images exhibit substantial variations in size, including buildings, cars, and roads. We introduce a scale-adaptive semantic label fusion strategy that enhances the segmentation of objects of varying sizes. We then introduce a novel cross-view instance label grouping strategy to mitigate the multi-view inconsistency problem in the 2D instance labels.
  arXiv  Detail & Related papers  (2024-03-18T14:15:39Z)
• Sat2Scene: 3D Urban Scene Generation from Satellite Images with Diffusion [77.34078223594686]
  We propose a novel architecture for direct 3D scene generation by introducing diffusion models into 3D sparse representations and combining them with neural rendering techniques. Specifically, our approach generates texture colors at the point level for a given geometry using a 3D diffusion model first, which is then transformed into a scene representation in a feed-forward manner. Experiments in two city-scale datasets show that our model demonstrates proficiency in generating photo-realistic street-view image sequences and cross-view urban scenes from satellite imagery.
  arXiv  Detail & Related papers  (2024-01-19T16:15:37Z)
• Holistic Inverse Rendering of Complex Facade via Aerial 3D Scanning [38.72679977945778]
  We use multi-view aerial images to reconstruct the geometry, lighting, and material of facades using neural signed distance fields (SDFs) The experiment demonstrates the superior quality of our method on facade holistic inverse rendering, novel view synthesis, and scene editing compared to state-of-the-art baselines.
  arXiv  Detail & Related papers  (2023-11-20T15:03:56Z)
• HORIZON: High-Resolution Semantically Controlled Panorama Synthesis [105.55531244750019]
  Panorama synthesis endeavors to craft captivating 360-degree visual landscapes, immersing users in the heart of virtual worlds. Recent breakthroughs in visual synthesis have unlocked the potential for semantic control in 2D flat images, but a direct application of these methods to panorama synthesis yields distorted content. We unveil an innovative framework for generating high-resolution panoramas, adeptly addressing the issues of spherical distortion and edge discontinuity through sophisticated spherical modeling.
  arXiv  Detail & Related papers  (2022-10-10T09:43:26Z)
• Urban Radiance Fields [77.43604458481637]
  We perform 3D reconstruction and novel view synthesis from data captured by scanning platforms commonly deployed for world mapping in urban outdoor environments. Our approach extends Neural Radiance Fields, which has been demonstrated to synthesize realistic novel images for small scenes in controlled settings. Each of these three extensions provides significant performance improvements in experiments on Street View data.
  arXiv  Detail & Related papers  (2021-11-29T15:58:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.