SpectralSplats: Robust Differentiable Tracking via Spectral Moment Supervision

• URL: http://arxiv.org/abs/2603.24036v1
• Date: Wed, 25 Mar 2026 07:48:22 GMT
• Title: SpectralSplats: Robust Differentiable Tracking via Spectral Moment Supervision
• Abstract summary: We introduce SpectralSplats, a robust tracking framework that shifts the gradient objective from the spatial to the frequency domain.<n>We demonstrate that SpectralSplats acts as a seamless, drop-in replacement for spatial losses across diverse deformation parameterizations.
• Score: 15.658046416731098
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: 3D Gaussian Splatting (3DGS) enables real-time, photorealistic novel view synthesis, making it a highly attractive representation for model-based video tracking. However, leveraging the differentiability of the 3DGS renderer "in the wild" remains notoriously fragile. A fundamental bottleneck lies in the compact, local support of the Gaussian primitives. Standard photometric objectives implicitly rely on spatial overlap; if severe camera misalignment places the rendered object outside the target's local footprint, gradients strictly vanish, leaving the optimizer stranded. We introduce SpectralSplats, a robust tracking framework that resolves this "vanishing gradient" problem by shifting the optimization objective from the spatial to the frequency domain. By supervising the rendered image via a set of global complex sinusoidal features (Spectral Moments), we construct a global basin of attraction, ensuring that a valid, directional gradient toward the target exists across the entire image domain, even when pixel overlap is completely nonexistent. To harness this global basin without introducing periodic local minima associated with high frequencies, we derive a principled Frequency Annealing schedule from first principles, gracefully transitioning the optimizer from global convexity to precise spatial alignment. We demonstrate that SpectralSplats acts as a seamless, drop-in replacement for spatial losses across diverse deformation parameterizations (from MLPs to sparse control points), successfully recovering complex deformations even from severely misaligned initializations where standard appearance-based tracking catastrophically fails.

Related papers

• ConeGaussian: Anti-Aliased Gaussian Ray-Tracing for Generic Central Cameras [56.46810752790842]
  We present ConeGaussian, a camera-model-agnostic anti-aliasing framework for Gaussian ray-based rendering.<n>We derive a closed-form response under a locally linear, depth-local, moment-matched approximation of the finite pixel footprint.<n>Unlike in mip-splatting, our scene-space frequency floor and filtering enable trivial composition at render time, allowing us to remove excess blurring.
  arXiv  Detail & Related papers  (2026-09-11T18:05:07Z)
• Visual Relocalization from Sparse Views in Aliased and Low-Texture Environments via Novel View Synthesis [17.82023714307905]
  We propose a visual relocalization method that departs from classical correspondence-based pipelines.<n>Our key contribution is a geometry-aware training strategy that combines photometric and geometric losses.<n>We show that this joint optimization produces a 3DGS model that better fits the underlying scene geometry.
  arXiv  Detail & Related papers  (2026-07-24T09:46:07Z)
• GrainGS: Gradient-Decoupled Gaussian Splatting for Efficient Dynamic Novel View Synthesis [94.20462261657525]
  GrainGS is a dynamic framework that combines a hierarchical anchor scaffold with per-Gaussian deformation.<n>It achieves high reconstruction quality, real-time novel view synthesis, and compact storage.
  arXiv  Detail & Related papers  (2026-07-23T15:52:35Z)
• RMGS-SLAM: Real-time Multi-sensor Gaussian Splatting SLAM [19.6725809458616]
  We present a tightly coupled LiDAR-Inertial-Visual 3DGS-based SLAM framework for real-time pose estimation and mapping.<n>The proposed method achieves a state of the art among real-time efficiency, localization accuracy, and rendering quality across diverse real-world scenes.
  arXiv  Detail & Related papers  (2026-04-14T16:36:11Z)
• COSMOS: Coherent Supergaussian Modeling with Spatial Priors for Sparse-View 3D Splatting [2.549594916296063]
  3D Gaussian Splatting (3DGS) has recently emerged as a promising approach for 3D reconstruction.<n>We propose Coherent supergaussian Modeling with Spatial Priors (COSMOS) to address this issue.<n>Our experiments on Blender and DTU show that COSMOS surpasses state-of-the-art methods in sparse-view settings without any external depth supervision.
  arXiv  Detail & Related papers  (2025-12-17T04:55:15Z)
• Robust 3DGS-based SLAM via Adaptive Kernel Smoothing [14.419840415480435]
  We argue that it is more critical to enhance robustness of the rendering process against parameter errors to ensure stable camera pose tracking.<n>Unlike conventional methods that focus solely on minimizing rendering error, our core insight is to make the rendering process more resilient to imperfections in the 3DGS parameters.<n>Our approach, while maintaining the overall quality of the scene reconstruction (mapping), significantly improves the robustness and accuracy of camera pose tracking.
  arXiv  Detail & Related papers  (2025-11-28T14:28:05Z)
• VA-GS: Enhancing the Geometric Representation of Gaussian Splatting via View Alignment [48.147381011235446]
  3D Gaussian Splatting has recently emerged as an efficient solution for real-time novel view synthesis.<n>We propose a novel method that enhances the geometric representation of 3D Gaussians through view alignment.<n>Our method achieves state-of-the-art performance in both surface reconstruction and novel view synthesis.
  arXiv  Detail & Related papers  (2025-10-13T14:44:50Z)
• GSFusion:Globally Optimized LiDAR-Inertial-Visual Mapping for Gaussian Splatting [20.665306154754564]
  We propose GSFusion, an online LiDAR-Inertial-Visual mapping system that ensures high-precision map consistency through a surfel-to-surfel constraint in the global pose-graph optimization.<n> Experiments on public and our datasets demonstrate our system outperforms existing 3DGS SLAM systems in terms of rendering quality and map-building efficiency.
  arXiv  Detail & Related papers  (2025-07-31T06:15:51Z)
• HoliGS: Holistic Gaussian Splatting for Embodied View Synthesis [59.25751939710903]
  We propose a novel deformable Gaussian splatting framework that addresses embodied view synthesis from long monocular RGB videos.<n>Our method leverages invertible Gaussian Splatting deformation networks to reconstruct large-scale, dynamic environments accurately.<n>Results highlight a practical and scalable solution for EVS in real-world scenarios.
  arXiv  Detail & Related papers  (2025-06-24T03:54:40Z)
• StableGS: A Floater-Free Framework for 3D Gaussian Splatting [9.935869165752283]
  3D Gaussian Splatting (3DGS) reconstructions are plagued by stubborn floater" artifacts that degrade their geometric and visual fidelity.<n>We propose StableGS, a novel framework that decouples geometric regularization from final appearance rendering.<n> Experiments on multiple benchmarks show StableGS not only eliminates floaters but also resolves the common blur-artifact trade-off.
  arXiv  Detail & Related papers  (2025-03-24T09:02:51Z)
• PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting [54.7468067660037]
  PF3plat sets a new state-of-the-art across all benchmarks, supported by comprehensive ablation studies validating our design choices.<n>Our framework capitalizes on fast speed, scalability, and high-quality 3D reconstruction and view synthesis capabilities of 3DGS.
  arXiv  Detail & Related papers  (2024-10-29T15:28:15Z)
• Improving Gaussian Splatting with Localized Points Management [52.009874685460694]
  Localized Point Management (LPM) is capable of identifying those error-contributing zones in greatest need for both point addition and geometry calibration.<n>LPM applies point densification in the identified zones and then reset the opacity of the points in front of these regions, creating a new opportunity to correct poorly conditioned points.<n> Notably, LPM improves both static 3DGS and dynamic SpaceTimeGS to achieve state-of-the-art rendering quality while retaining real-time speeds.
  arXiv  Detail & Related papers  (2024-06-06T16:55:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.