Toy-GS: Assembling Local Gaussians for Precisely Rendering Large-Scale Free Camera Trajectories

• URL: http://arxiv.org/abs/2412.10078v1
• Date: Fri, 13 Dec 2024 12:10:53 GMT
• Title: Toy-GS: Assembling Local Gaussians for Precisely Rendering Large-Scale Free Camera Trajectories
• Authors: Xiaohan Zhang, Zhenyu Sun, Yukui Qiu, Junyan Su, Qi Liu,
• Abstract summary: Currently, 3D rendering for large-scale free camera trajectories, namely, arbitrary input camera trajectories, poses significant challenges.<n>This paper presents a Toy-GS method for accurately rendering large-scale free camera trajectories.
• Score: 9.953178002894877
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Currently, 3D rendering for large-scale free camera trajectories, namely, arbitrary input camera trajectories, poses significant challenges: 1) The distribution and observation angles of the cameras are irregular, and various types of scenes are included in the free trajectories; 2) Processing the entire point cloud and all images at once for large-scale scenes requires a substantial amount of GPU memory. This paper presents a Toy-GS method for accurately rendering large-scale free camera trajectories. Specifically, we propose an adaptive spatial division approach for free trajectories to divide cameras and the sparse point cloud of the entire scene into various regions according to camera poses. Training each local Gaussian in parallel for each area enables us to concentrate on texture details and minimize GPU memory usage. Next, we use the multi-view constraint and position-aware point adaptive control (PPAC) to improve the rendering quality of texture details. In addition, our regional fusion approach combines local and global Gaussians to enhance rendering quality with an increasing number of divided areas. Extensive experiments have been carried out to confirm the effectiveness and efficiency of Toy-GS, leading to state-of-the-art results on two public large-scale datasets as well as our SCUTic dataset. Our proposal demonstrates an enhancement of 1.19 dB in PSNR and conserves 7 G of GPU memory when compared to various benchmarks.

Related papers

• PCR-GS: COLMAP-Free 3D Gaussian Splatting via Pose Co-Regularizations [102.0476991174456]
  COLMAP-free 3DGS has attracted increasing attention due to its remarkable performance in reconstructing high-quality 3D scenes from unposed images or videos.<n>We propose PCR-GS, an innovative COLMAP-free 3DGS technique that achieves superior 3D scene modeling and camera pose estimation via camera pose co-regularization.
  arXiv  Detail & Related papers  (2025-07-18T13:09:33Z)
• TraGraph-GS: Trajectory Graph-based Gaussian Splatting for Arbitrary Large-Scale Scene Rendering [22.856356474506168]
  Existing methods typically partition large scenes into multiple regions, reconstruct a 3D representation using Gaussian splatting for each region, and eventually merge them for novel view rendering.<n>We propose TraGraph-GS, leveraging a trajectory graph to enable high-precision rendering for arbitrarily large-scale scenes.<n>Our method achieves an average improvement of 1.86 dB in PSNR on aerial datasets and 1.62 dB on ground datasets compared to state-of-the-art approaches.
  arXiv  Detail & Related papers  (2025-06-10T11:43:04Z)
• On-the-fly Reconstruction for Large-Scale Novel View Synthesis from Unposed Images [48.8544345503807]
  We present an on-the-fly method to produce camera poses and a trained 3DGS immediately after capture.<n>Our method can handle dense and wide-baseline captures of ordered photo sequences and large-scale scenes.
  arXiv  Detail & Related papers  (2025-06-05T20:10:18Z)
• VTGaussian-SLAM: RGBD SLAM for Large Scale Scenes with Splatting View-Tied 3D Gaussians [27.62796825514193]
  State-of-the-art methods employ 3D Gaussians to represent a scene, and render these Gaussians through splatting for higher efficiency and better rendering.<n>These methods cannot scale up to extremely large scenes, due to the inefficient tracking and mapping strategies.<n>To resolve this issue, we propose novel tracking and mapping strategies to work with a novel 3D representation, dubbed view-tied 3D Gaussians.
  arXiv  Detail & Related papers  (2025-06-03T10:59:19Z)
• LODGE: Level-of-Detail Large-Scale Gaussian Splatting with Efficient Rendering [68.93333348474988]
  We present a novel level-of-detail (LOD) method for 3D Gaussian Splatting on memory-constrained devices.<n>Our approach iteratively selects optimal subsets of Gaussians based on camera distance.<n>Our method achieves state-of-the-art performance on both outdoor (Hierarchical 3DGS) and indoor (Zip-NeRF) datasets.
  arXiv  Detail & Related papers  (2025-05-29T06:50:57Z)
• FRAME: Floor-aligned Representation for Avatar Motion from Egocentric Video [52.33896173943054]
  Egocentric motion capture with a head-mounted body-facing stereo camera is crucial for VR and AR applications. Existing methods rely on synthetic pretraining and struggle to generate smooth and accurate predictions in real-world settings. We propose FRAME, a simple yet effective architecture that combines device pose and camera feeds for state-of-the-art body pose prediction.
  arXiv  Detail & Related papers  (2025-03-29T14:26:06Z)
• TrackGS: Optimizing COLMAP-Free 3D Gaussian Splatting with Global Track Constraints [40.9371798496134]
  We introduce TrackGS, which incorporates feature tracks to globally constrain multi-view geometry. We also propose minimizing both reprojection and backprojection errors for better geometric consistency. By deriving the gradient of intrinsics, we unify camera parameter estimation with 3DGS training into a joint optimization framework.
  arXiv  Detail & Related papers  (2025-02-27T06:16:04Z)
• GS-Blur: A 3D Scene-Based Dataset for Realistic Image Deblurring [50.72230109855628]
  We propose GS-Blur, a dataset of synthesized realistic blurry images created using a novel approach. We first reconstruct 3D scenes from multi-view images using 3D Gaussian Splatting (3DGS), then render blurry images by moving the camera view along the randomly generated motion trajectories. By adopting various camera trajectories in reconstructing our GS-Blur, our dataset contains realistic and diverse types of blur, offering a large-scale dataset that generalizes well to real-world blur.
  arXiv  Detail & Related papers  (2024-10-31T06:17:16Z)
• GSplatLoc: Grounding Keypoint Descriptors into 3D Gaussian Splatting for Improved Visual Localization [1.4466437171584356]
  3D Gaussian Splatting (3DGS) allows for the compact encoding of both 3D geometry and scene appearance with its spatial features. We propose distilling dense keypoint descriptors into 3DGS to improve the model's spatial understanding. Our approach surpasses state-of-the-art Neural Render Pose (NRP) methods, including NeRFMatch and PNeRFLoc.
  arXiv  Detail & Related papers  (2024-09-24T23:18:32Z)
• PUP 3D-GS: Principled Uncertainty Pruning for 3D Gaussian Splatting [59.277480452459315]
  We propose a principled sensitivity pruning score that preserves visual fidelity and foreground details at significantly higher compression ratios.<n>We also propose a multi-round prune-refine pipeline that can be applied to any pretrained 3D-GS model without changing its training pipeline.
  arXiv  Detail & Related papers  (2024-06-14T17:53:55Z)
• Dual-Camera Smooth Zoom on Mobile Phones [55.4114152554769]
  We introduce a new task, ie, dual-camera smooth zoom (DCSZ) to achieve a smooth zoom preview. The frame models (FI) technique is a potential solution but struggles with ground-truth collection. We suggest a data factory solution where continuous virtual cameras are assembled to generate DCSZ data by rendering reconstructed 3D models of the scene.
  arXiv  Detail & Related papers  (2024-04-07T10:28:01Z)
• HO-Gaussian: Hybrid Optimization of 3D Gaussian Splatting for Urban Scenes [24.227745405760697]
  We propose a hybrid optimization method named HO-Gaussian, which combines a grid-based volume with the 3DGS pipeline. Results on widely used autonomous driving datasets demonstrate that HO-Gaussian achieves photo-realistic rendering in real-time on multi-camera urban datasets.
  arXiv  Detail & Related papers  (2024-03-29T07:58:21Z)
• VastGaussian: Vast 3D Gaussians for Large Scene Reconstruction [59.40711222096875]
  We present VastGaussian, the first method for high-quality reconstruction and real-time rendering on large scenes based on 3D Gaussian Splatting. Our approach outperforms existing NeRF-based methods and achieves state-of-the-art results on multiple large scene datasets.
  arXiv  Detail & Related papers  (2024-02-27T11:40:50Z)
• Gaussian-SLAM: Photo-realistic Dense SLAM with Gaussian Splatting [24.160436463991495]
  We present a dense simultaneous localization and mapping (SLAM) method that uses 3D Gaussians as a scene representation. Our approach enables interactive-time reconstruction and photo-realistic rendering from real-world single-camera RGBD videos.
  arXiv  Detail & Related papers  (2023-12-06T10:47:53Z)
• Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering [71.44349029439944]
  Recent 3D Gaussian Splatting method has achieved the state-of-the-art rendering quality and speed. We introduce Scaffold-GS, which uses anchor points to distribute local 3D Gaussians. We show that our method effectively reduces redundant Gaussians while delivering high-quality rendering.
  arXiv  Detail & Related papers  (2023-11-30T17:58:57Z)
• Sampling Based Scene-Space Video Processing [89.49726406622842]
  We present a novel, sampling-based framework for processing video. It enables high-quality scene-space video effects in the presence of inevitable errors in depth and camera pose estimation. We present results for various casually captured, hand-held, moving, compressed, monocular videos.
  arXiv  Detail & Related papers  (2021-02-05T05:55:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.