Cross-Spectral Stereo Inertial Odometry

• URL: http://arxiv.org/abs/2606.29757v1
• Date: Mon, 29 Jun 2026 04:00:59 GMT
• Title: Cross-Spectral Stereo Inertial Odometry
• Abstract summary: We present an asynchronous real-time cross-spectral visual-thermal-inertial (VTI) system that temporally decouples high-latency deep matching from high-rate state estimation.<n>Our system overcomes spectral redundancy, yielding superior accuracy in nominal daylight while ensuring robustness in visually degraded environments.
• Score: 6.602180427145135
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Standard stereo VIO focuses exclusively on the benefit of metric scale via single-spectrum baselines, often overlooking the risks of spectral redundancy. This structural limitation leads to correlated failures, where both sensors simultaneously fail in degraded environments that affect their shared spectrum. Leveraging a cross-spectral system presents a complementary solution to this issue, yet the significant appearance gap between modalities renders standard matching ineffective. Existing deep learning-based matchers, while effective, introduce inference latencies that violate real-time constraints. To bridge this gap, we present an asynchronous real-time cross-spectral visual-thermal-inertial (VTI) system that temporally decouples high-latency deep matching from high-rate state estimation. Our architecture incorporates a spectral-aware weighting scheme that dynamically balances modality reliance based on photometric entropy and thermal noise, ensuring robustness against both abrupt lighting changes and thermal artifacts. Furthermore, we introduce a seamless handling mechanism for thermal Non-uniformity Correction (NUC) to maintain tracking continuity. Extensive experiments across diverse scenarios confirm that our system overcomes spectral redundancy, yielding superior accuracy in nominal daylight while ensuring robustness in visually degraded environments. We will open source our code and data: https://github.com/seungsang07/cross-spectral-stereo-inertial-odometry

Related papers

• Weaving Light and Time: Unified Harmonic-Geometric Representation Learning for Dense RGB-Event Parsing [55.13802890769086]
  We present Evita, the first unified backbone specifically engineered for dedicated dense RGB-Event parsing.<n>To achieve profound modal synergy, Evita embeds a suite of intrinsic co-learning modules directly into every encoder layer.<n>Evita establishes new state-of-the-art metrics while delivering a superior accuracy-latency trade-off for real-time multimodal perception.
  arXiv  Detail & Related papers  (2026-07-10T06:52:09Z)
• A Benchmark for Heterogeneous Stereo Deblurring with Physically- and Epipolar-constrained Cross Attention [3.373984146148716]
  We present a dedicated framework for heterogeneous stereo deblurring.<n>We propose physically- and epipolar-constrained cross attention (PECA)<n>PECA restricts cross-view matching to an epipolar search window bounded by a optics-derived disparity upper bound.
  arXiv  Detail & Related papers  (2026-06-24T15:34:20Z)
• Set-Based Transformer for Atmospheric Compensation in Standoff LWIR Hyperspectral Imaging [6.066442015301664]
  We present a lightweight set-based deep learning framework that takes multiple radiance measurements and estimates transmittance, atmospheric path radiance, and a shared downwelling spectrum.<n>We analyze the learned representation with a sparse autoencoder and observe that several latent features do activate on geographically coherent subsets of the test data.
  arXiv  Detail & Related papers  (2026-06-06T20:25:30Z)
• LISA: Language-guided Interference-aware Spatial-Frequency Attention for Driver Gaze Estimation [12.034295331199813]
  Driver gaze estimation serves as a fundamental metric for evaluating driver attentiveness in modern monitoring systems.<n>We propose LISA, a textbfLanguage-guided textbfInterference-aware textbfSpatial-Frequency textbfAttention framework.
  arXiv  Detail & Related papers  (2026-05-17T06:57:31Z)
• ZoomSpec: A Physics-Guided Coarse-to-Fine Framework for Wideband Spectrum Sensing [7.560433707709695]
  ZoomSpec is a physics-guided coarse-to-fine framework integrating signal processing priors with deep learning.<n> Evaluations on the SpaceNet real-world dataset demonstrate state-of-the-art 78.1 mAP@0.5:0.95.
  arXiv  Detail & Related papers  (2026-04-15T07:29:25Z)
• Robust Graph Representation Learning via Adaptive Spectral Contrast [0.9558392439655014]
  High-frequency signals are indispensable for encoding heterophily but exhibit significantly higher variance under spectrally concentrated perturbations.<n>We propose ASPECT, a framework that resolves this dilemma through a reliability-aware spectral gating mechanism.<n>ASPECT achieves new state-of-the-art performance on 8 out of 9 benchmarks, effectively decoupling meaningful structural heterophily from incidental noise.
  arXiv  Detail & Related papers  (2026-04-02T10:38:00Z)
• Parallel Complex Diffusion for Scalable Time Series Generation [50.01609741902786]
  PaCoDi is a spectral-native architecture that decouples generative modeling in the frequency domain.<n>We show that PaCoDi outperforms existing baselines in both generation quality and inference speed.
  arXiv  Detail & Related papers  (2026-02-10T14:31:53Z)
• Strip-Fusion: Spatiotemporal Fusion for Multispectral Pedestrian Detection [0.27528170226206433]
  Multispectral modalities (visible light and thermal) can boost pedestrian detection performance by providing complementary visual information.<n>Existing approaches primarily focus on spatial fusion and often neglect temporal information.<n>This work proposes Strip-Fusion, a spatial-temporal fusion network that is robust to misalignment in input images.
  arXiv  Detail & Related papers  (2026-01-25T21:58:07Z)
• HAD: Hierarchical Asymmetric Distillation to Bridge Spatio-Temporal Gaps in Event-Based Object Tracking [80.07224739976911]
  Event cameras offer exceptional temporal resolution and a range (modal)<n> RGB cameras excel at capturing rich texture with high resolution, whereas event cameras offer exceptional temporal resolution and a range (modal)
  arXiv  Detail & Related papers  (2025-10-22T13:15:13Z)
• Adaptive Stereo Depth Estimation with Multi-Spectral Images Across All Lighting Conditions [58.88917836512819]
  We propose a novel framework incorporating stereo depth estimation to enforce accurate geometric constraints. To mitigate the effects of poor lighting on stereo matching, we introduce Degradation Masking. Our method achieves state-of-the-art (SOTA) performance on the Multi-Spectral Stereo (MS2) dataset.
  arXiv  Detail & Related papers  (2024-11-06T03:30:46Z)
• Low-Light Video Enhancement via Spatial-Temporal Consistent Decomposition [52.89441679581216]
  Low-Light Video Enhancement (LLVE) seeks to restore dynamic or static scenes plagued by severe invisibility and noise.<n>We present an innovative video decomposition strategy that incorporates view-independent and view-dependent components.<n>Our framework consistently outperforms existing methods, establishing a new SOTA performance.
  arXiv  Detail & Related papers  (2024-05-24T15:56:40Z)
• ESSAformer: Efficient Transformer for Hyperspectral Image Super-resolution [76.7408734079706]
  Single hyperspectral image super-resolution (single-HSI-SR) aims to restore a high-resolution hyperspectral image from a low-resolution observation. We propose ESSAformer, an ESSA attention-embedded Transformer network for single-HSI-SR with an iterative refining structure.
  arXiv  Detail & Related papers  (2023-07-26T07:45:14Z)
• Degradation-agnostic Correspondence from Resolution-asymmetric Stereo [96.03964515969652]
  We study the problem of stereo matching from a pair of images with different resolutions, e.g., those acquired with a tele-wide camera system. We propose to impose the consistency between two views in a feature space instead of the image space, named feature-metric consistency. We find that, although a stereo matching network trained with the photometric loss is not optimal, its feature extractor can produce degradation-agnostic and matching-specific features.
  arXiv  Detail & Related papers  (2022-04-04T12:24:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.