BRIGHT-VO: Brightness-Guided Hybrid Transformer for Visual Odometry with Multi-modality Refinement Module

• URL: http://arxiv.org/abs/2501.08659v2
• Date: Thu, 16 Jan 2025 03:51:49 GMT
• Title: BRIGHT-VO: Brightness-Guided Hybrid Transformer for Visual Odometry with Multi-modality Refinement Module
• Authors: Dongzhihan Wang, Yang Yang, Liang Xu,
• Abstract summary: Visual odometry (VO) plays a crucial role in autonomous driving, robotic navigation, and other related tasks. We introduce BrightVO, a novel VO model based on Transformer architecture, which performs front-end visual feature extraction. Using pose graph optimization, this module iteratively refines pose estimates to reduce errors and improve both accuracy and robustness.
• Score: 11.898515581215708
• License:
• Abstract: Visual odometry (VO) plays a crucial role in autonomous driving, robotic navigation, and other related tasks by estimating the position and orientation of a camera based on visual input. Significant progress has been made in data-driven VO methods, particularly those leveraging deep learning techniques to extract image features and estimate camera poses. However, these methods often struggle in low-light conditions because of the reduced visibility of features and the increased difficulty of matching keypoints. To address this limitation, we introduce BrightVO, a novel VO model based on Transformer architecture, which not only performs front-end visual feature extraction, but also incorporates a multi-modality refinement module in the back-end that integrates Inertial Measurement Unit (IMU) data. Using pose graph optimization, this module iteratively refines pose estimates to reduce errors and improve both accuracy and robustness. Furthermore, we create a synthetic low-light dataset, KiC4R, which includes a variety of lighting conditions to facilitate the training and evaluation of VO frameworks in challenging environments. Experimental results demonstrate that BrightVO achieves state-of-the-art performance on both the KiC4R dataset and the KITTI benchmarks. Specifically, it provides an average improvement of 20% in pose estimation accuracy in normal outdoor environments and 259% in low-light conditions, outperforming existing methods. For widespread use and further development, the research work is fully open-source at https://github.com/Anastasiawd/BrightVO.

Related papers

• LIR-LIVO: A Lightweight,Robust LiDAR/Vision/Inertial Odometry with Illumination-Resilient Deep Features [8.095827028713684]
  The proposed method leverages deep learning-based illumination-resilient features and LiDAR-Inertial-Visual Odometry (LIVO) LIR-LIVO achieves state-of-the-art (SOTA) accuracy and robustness with low computational cost.
  arXiv  Detail & Related papers  (2025-02-12T05:28:10Z)
• XR-VIO: High-precision Visual Inertial Odometry with Fast Initialization for XR Applications [34.2082611110639]
  This paper presents a novel approach to Visual Inertial Odometry (VIO) focusing on the initialization and feature matching modules. Existing methods for gyroscopes often suffer from poor stability in visual Structure from Motion (SfM) or in solving a huge number of parameters simultaneously. By tightly coupling measurements, we enhance the robustness and accuracy of visual SfM. In terms of feature matching, we introduce a hybrid method that combines optical flow and descriptor-based matching.
  arXiv  Detail & Related papers  (2025-02-03T12:17:51Z)
• Visual Fourier Prompt Tuning [63.66866445034855]
  We propose the Visual Fourier Prompt Tuning (VFPT) method as a general and effective solution for adapting large-scale transformer-based models. Our approach incorporates the Fast Fourier Transform into prompt embeddings and harmoniously considers both spatial and frequency domain information. Our results demonstrate that our approach outperforms current state-of-the-art baselines on two benchmarks.
  arXiv  Detail & Related papers  (2024-11-02T18:18:35Z)
• Efficient Camera Exposure Control for Visual Odometry via Deep Reinforcement Learning [10.886819238167286]
  This study employs a deep reinforcement learning framework to train agents for exposure control. A lightweight image simulator is developed to facilitate the training process. Different levels of reward functions are crafted to enhance the VO systems.
  arXiv  Detail & Related papers  (2024-08-30T04:37:52Z)
• BVI-RLV: A Fully Registered Dataset and Benchmarks for Low-Light Video Enhancement [56.97766265018334]
  This paper introduces a low-light video dataset, consisting of 40 scenes with various motion scenarios under two distinct low-lighting conditions. We provide fully registered ground truth data captured in normal light using a programmable motorized dolly and refine it via an image-based approach for pixel-wise frame alignment across different light levels. Our experimental results demonstrate the significance of fully registered video pairs for low-light video enhancement (LLVE) and the comprehensive evaluation shows that the models trained with our dataset outperform those trained with the existing datasets.
  arXiv  Detail & Related papers  (2024-07-03T22:41:49Z)
• Enhancing Low-light Light Field Images with A Deep Compensation Unfolding Network [52.77569396659629]
  This paper presents the deep compensation network unfolding (DCUNet) for restoring light field (LF) images captured under low-light conditions. The framework uses the intermediate enhanced result to estimate the illumination map, which is then employed in the unfolding process to produce a new enhanced result. To properly leverage the unique characteristics of LF images, this paper proposes a pseudo-explicit feature interaction module.
  arXiv  Detail & Related papers  (2023-08-10T07:53:06Z)
• Ultra-High-Definition Low-Light Image Enhancement: A Benchmark and Transformer-Based Method [51.30748775681917]
  We consider the task of low-light image enhancement (LLIE) and introduce a large-scale database consisting of images at 4K and 8K resolution. We conduct systematic benchmarking studies and provide a comparison of current LLIE algorithms. As a second contribution, we introduce LLFormer, a transformer-based low-light enhancement method.
  arXiv  Detail & Related papers  (2022-12-22T09:05:07Z)
• Toward Fast, Flexible, and Robust Low-Light Image Enhancement [87.27326390675155]
  We develop a new Self-Calibrated Illumination (SCI) learning framework for fast, flexible, and robust brightening images in real-world low-light scenarios. Considering the computational burden of the cascaded pattern, we construct the self-calibrated module which realizes the convergence between results of each stage. We make comprehensive explorations to SCI's inherent properties including operation-insensitive adaptability and model-irrelevant generality.
  arXiv  Detail & Related papers  (2022-04-21T14:40:32Z)
• Towards Scale Consistent Monocular Visual Odometry by Learning from the Virtual World [83.36195426897768]
  We propose VRVO, a novel framework for retrieving the absolute scale from virtual data. We first train a scale-aware disparity network using both monocular real images and stereo virtual data. The resulting scale-consistent disparities are then integrated with a direct VO system.
  arXiv  Detail & Related papers  (2022-03-11T01:51:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.