DirectFisheye-GS: Enabling Native Fisheye Input in Gaussian Splatting with Cross-View Joint Optimization

• URL: http://arxiv.org/abs/2604.00648v1
• Date: Wed, 01 Apr 2026 09:00:04 GMT
• Title: DirectFisheye-GS: Enabling Native Fisheye Input in Gaussian Splatting with Cross-View Joint Optimization
• Abstract summary: 3D Gaussian Splatting (3DGS) has enabled efficient 3D scene reconstruction from everyday images with real-time, high-fidelity rendering.<n>Fisheye cameras, with their wider field of view (FOV), promise high-quality reconstructions from fewer inputs.
• Score: 17.91043282211744
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: 3D Gaussian Splatting (3DGS) has enabled efficient 3D scene reconstruction from everyday images with real-time, high-fidelity rendering, greatly advancing VR/AR applications. Fisheye cameras, with their wider field of view (FOV), promise high-quality reconstructions from fewer inputs and have recently attracted much attention. However, since 3DGS relies on rasterization, most subsequent works involving fisheye camera inputs first undistort images before training, which introduces two problems: 1) Black borders at image edges cause information loss and negate the fisheye's large FOV advantage; 2) Undistortion's stretch-and-interpolate resampling spreads each pixel's value over a larger area, diluting detail density -- causes 3DGS overfitting these low-frequency zones, producing blur and floating artifacts. In this work, we integrate fisheye camera model into the original 3DGS framework, enabling native fisheye image input for training without preprocessing. Despite correct modeling, we observed that the reconstructed scenes still exhibit floaters at image edges: Distortion increases toward the periphery, and 3DGS's original per-iteration random-selecting-view optimization ignores the cross-view correlations of a Gaussian, leading to extreme shapes (e.g., oversized or elongated) that degrade reconstruction quality. To address this, we introduce a feature-overlap-driven cross-view joint optimization strategy that establishes consistent geometric and photometric constraints across views-a technique equally applicable to existing pinhole-camera-based pipelines. Our DirectFisheye-GS matches or surpasses state-of-the-art performance on public datasets.

Related papers

• GaussVid: Sparse-View Gaussian Splatting with 3D-Aware Video Diffusion Priors [14.045684695919116]
  3DGS has achieved remarkable success in synthesis; however, reconstructions under sparse views often exhibit noticeable artifacts.<n>We propose a novel 3D-aware video restoration framework designed to enhance the quality of sparse 3DGS reconstruction.<n>Our approach attains the best pixel- and structure-level fidelity (PSNR/SSIM) and improves multi-view consistency.
  arXiv  Detail & Related papers  (2026-08-22T08:47:29Z)
• Visual Relocalization from Sparse Views in Aliased and Low-Texture Environments via Novel View Synthesis [17.82023714307905]
  We propose a visual relocalization method that departs from classical correspondence-based pipelines.<n>Our key contribution is a geometry-aware training strategy that combines photometric and geometric losses.<n>We show that this joint optimization produces a 3DGS model that better fits the underlying scene geometry.
  arXiv  Detail & Related papers  (2026-07-24T09:46:07Z)
• PointSplat: Efficient Geometry-Driven Pruning and Transformer Refinement for 3D Gaussian Splatting [2.5823344978857388]
  3D Gaussian Splatting has recently unlocked real-time, high-fidelity novel view synthesis by representing scenes using explicit 3D primitives.<n>Traditional methods often require millions of Gaussians to capture complex scenes, leading to significant memory and storage demands.<n>Recent approaches have addressed this issue through pruning and per-scene fine-tuning of Gaussian parameters, thereby reducing the model size while maintaining visual quality.
  arXiv  Detail & Related papers  (2026-04-10T20:56:18Z)
• Fisheye3R: Adapting Unified 3D Feed-Forward Foundation Models to Fisheye Lenses [5.987299558427844]
  Fisheye3R is a novel adaptation framework that extends multi-view 3D reconstruction foundation models to accommodate fisheye inputs without performance regression on perspective images.<n>Our approach consistently improves camera pose, depth, point map, and field-of-view estimation on fisheye images.
  arXiv  Detail & Related papers  (2026-03-30T18:18:56Z)
• PrismGS: Physically-Grounded Anti-Aliasing for High-Fidelity Large-Scale 3D Gaussian Splatting [23.277915756682944]
  3D Gaussian Splatting (3DGS) has recently enabled real-time rendering in compact scenes, but scaling to large urban environments introduces severe aliasing artifacts.<n>We propose PrismGS, a physically-grounded regularization framework that improves the intrinsic rendering behavior of 3D Gaussians.
  arXiv  Detail & Related papers  (2025-10-09T06:21:45Z)
• Revisiting Depth Representations for Feed-Forward 3D Gaussian Splatting [57.43483622778394]
  We introduce PM-Loss, a novel regularization loss based on a pointmap predicted by a pre-trained transformer.<n>With the improved depth map, our method significantly improves the feed-forward 3DGS across various architectures and scenes.
  arXiv  Detail & Related papers  (2025-06-05T17:58:23Z)
• DoF-Gaussian: Controllable Depth-of-Field for 3D Gaussian Splatting [52.52398576505268]
  We introduce DoF-Gaussian, a controllable depth-of-field method for 3D-GS.<n>We develop a lens-based imaging model based on geometric optics principles to control DoF effects.<n>Our framework is customizable and supports various interactive applications.
  arXiv  Detail & Related papers  (2025-03-02T05:57:57Z)
• Beyond Gaussians: Fast and High-Fidelity 3D Splatting with Linear Kernels [51.08794269211701]
  We introduce 3D Linear Splatting (3DLS), which replaces Gaussian kernels with linear kernels to achieve sharper and more precise results.<n>3DLS demonstrates state-of-the-art fidelity and accuracy, along with a 30% FPS improvement over baseline 3DGS.
  arXiv  Detail & Related papers  (2024-11-19T11:59:54Z)
• Fisheye-GS: Lightweight and Extensible Gaussian Splatting Module for Fisheye Cameras [17.75361921797657]
  We introduce Fisheye-GS.This innovative method recalculates the projection transformation and its gradients for fisheye cameras. Our approach can be seamlessly integrated as a module into other efficient 3D rendering methods.
  arXiv  Detail & Related papers  (2024-09-07T07:53:40Z)
• PUP 3D-GS: Principled Uncertainty Pruning for 3D Gaussian Splatting [59.277480452459315]
  We propose a principled sensitivity pruning score that preserves visual fidelity and foreground details at significantly higher compression ratios.<n>We also propose a multi-round prune-refine pipeline that can be applied to any pretrained 3D-GS model without changing its training pipeline.
  arXiv  Detail & Related papers  (2024-06-14T17:53:55Z)
• VastGaussian: Vast 3D Gaussians for Large Scene Reconstruction [59.40711222096875]
  We present VastGaussian, the first method for high-quality reconstruction and real-time rendering on large scenes based on 3D Gaussian Splatting. Our approach outperforms existing NeRF-based methods and achieves state-of-the-art results on multiple large scene datasets.
  arXiv  Detail & Related papers  (2024-02-27T11:40:50Z)
• Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering [71.44349029439944]
  Recent 3D Gaussian Splatting method has achieved the state-of-the-art rendering quality and speed. We introduce Scaffold-GS, which uses anchor points to distribute local 3D Gaussians. We show that our method effectively reduces redundant Gaussians while delivering high-quality rendering.
  arXiv  Detail & Related papers  (2023-11-30T17:58:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.