FlexDrive: Toward Trajectory Flexibility in Driving Scene Reconstruction and Rendering

• URL: http://arxiv.org/abs/2502.21093v2
• Date: Mon, 03 Mar 2025 03:48:47 GMT
• Title: FlexDrive: Toward Trajectory Flexibility in Driving Scene Reconstruction and Rendering
• Authors: Jingqiu Zhou, Lue Fan, Linjiang Huang, Xiaoyu Shi, Si Liu, Zhaoxiang Zhang, Hongsheng Li,
• Abstract summary: We introduce an Inverse View Warping technique to create compact and high-quality images as supervision for the reconstruction of the out-of-path views.<n>Our method achieves superior in-path and out-of-path reconstruction and rendering performance on the widely used Open dataset.
• Score: 79.39246982782717
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Driving scene reconstruction and rendering have advanced significantly using the 3D Gaussian Splatting. However, most prior research has focused on the rendering quality along a pre-recorded vehicle path and struggles to generalize to out-of-path viewpoints, which is caused by the lack of high-quality supervision in those out-of-path views. To address this issue, we introduce an Inverse View Warping technique to create compact and high-quality images as supervision for the reconstruction of the out-of-path views, enabling high-quality rendering results for those views. For accurate and robust inverse view warping, a depth bootstrap strategy is proposed to obtain on-the-fly dense depth maps during the optimization process, overcoming the sparsity and incompleteness of LiDAR depth data. Our method achieves superior in-path and out-of-path reconstruction and rendering performance on the widely used Waymo Open dataset. In addition, a simulator-based benchmark is proposed to obtain the out-of-path ground truth and quantitatively evaluate the performance of out-of-path rendering, where our method outperforms previous methods by a significant margin.

Related papers

• Rendering Anywhere You See: Renderability Field-guided Gaussian Splatting [4.89907242398523]
  We propose renderability field-guided gaussian splatting (RF-GS) for scene view synthesis. RF-GS quantifies input inhomogeneity through a renderability field, guiding pseudo-view sampling to enhanced visual consistency. Our experiments on simulated and real-world data show that our method outperforms existing approaches in rendering stability.
  arXiv  Detail & Related papers  (2025-04-27T14:41:01Z)
• Horizon-GS: Unified 3D Gaussian Splatting for Large-Scale Aerial-to-Ground Scenes [55.15494682493422]
  We introduce Horizon-GS, a novel approach built upon Gaussian Splatting techniques, to tackle the unified reconstruction and rendering for aerial and street views.<n>Our method addresses the key challenges of combining these perspectives with a new training strategy, overcoming viewpoint discrepancies to generate high-fidelity scenes.
  arXiv  Detail & Related papers  (2024-12-02T17:42:00Z)
• Drone-assisted Road Gaussian Splatting with Cross-view Uncertainty [10.37108303188536]
  3D Gaussian Splatting (3D-GS) has made groundbreaking progress in neural rendering. The general fidelity of large-scale road scene renderings is often limited by the input imagery. We introduce the cross-view uncertainty to 3D-GS by matching the car-view ensemble-based rendering uncertainty to aerial images.
  arXiv  Detail & Related papers  (2024-08-27T17:59:55Z)
• Efficient Depth-Guided Urban View Synthesis [52.841803876653465]
  We introduce Efficient Depth-Guided Urban View Synthesis (EDUS) for fast feed-forward inference and efficient per-scene fine-tuning. EDUS exploits noisy predicted geometric priors as guidance to enable generalizable urban view synthesis from sparse input images. Our results indicate that EDUS achieves state-of-the-art performance in sparse view settings when combined with fast test-time optimization.
  arXiv  Detail & Related papers  (2024-07-17T08:16:25Z)
• Pixel to Elevation: Learning to Predict Elevation Maps at Long Range using Images for Autonomous Offroad Navigation [10.898724668444125]
  We present a learning-based approach capable of predicting terrain elevation maps at long-range using only onboard egocentric images in real-time. We experimentally validate the applicability of our proposed approach for autonomous offroad robotic navigation in complex and unstructured terrain.
  arXiv  Detail & Related papers  (2024-01-30T22:37:24Z)
• HarmonicNeRF: Geometry-Informed Synthetic View Augmentation for 3D Scene Reconstruction in Driving Scenarios [2.949710700293865]
  HarmonicNeRF is a novel approach for outdoor self-supervised monocular scene reconstruction. It capitalizes on the strengths of NeRF and enhances surface reconstruction accuracy by augmenting the input space with geometry-informed synthetic views. Our approach establishes new benchmarks in synthesizing novel depth views and reconstructing scenes, significantly outperforming existing methods.
  arXiv  Detail & Related papers  (2023-10-09T07:42:33Z)
• Monocular BEV Perception of Road Scenes via Front-to-Top View Projection [57.19891435386843]
  We present a novel framework that reconstructs a local map formed by road layout and vehicle occupancy in the bird's-eye view. Our model runs at 25 FPS on a single GPU, which is efficient and applicable for real-time panorama HD map reconstruction.
  arXiv  Detail & Related papers  (2022-11-15T13:52:41Z)
• Structure-Aware Completion of Photogrammetric Meshes in Urban Road Environment [13.7725733099315]
  The paper proposes a structure-aware completion approach to improve the quality of meshes by removing undesired vehicles on the road seamlessly. It should be noted that the proposed method is also capable to handle tiled mesh models for large-scale scenes.
  arXiv  Detail & Related papers  (2020-11-23T05:04:28Z)
• Photon-Driven Neural Path Guiding [102.12596782286607]
  We present a novel neural path guiding approach that can reconstruct high-quality sampling distributions for path guiding from a sparse set of samples. We leverage photons traced from light sources as the input for sampling density reconstruction, which is highly effective for challenging scenes with strong global illumination. Our approach achieves significantly better rendering results of testing scenes than previous state-of-the-art path guiding methods.
  arXiv  Detail & Related papers  (2020-10-05T04:54:01Z)
• Deep 3D Capture: Geometry and Reflectance from Sparse Multi-View Images [59.906948203578544]
  We introduce a novel learning-based method to reconstruct the high-quality geometry and complex, spatially-varying BRDF of an arbitrary object. We first estimate per-view depth maps using a deep multi-view stereo network. These depth maps are used to coarsely align the different views. We propose a novel multi-view reflectance estimation network architecture.
  arXiv  Detail & Related papers  (2020-03-27T21:28:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.