Efficient Camera Exposure Control for Visual Odometry via Deep Reinforcement Learning

• URL: http://arxiv.org/abs/2408.17005v1
• Date: Fri, 30 Aug 2024 04:37:52 GMT
• Title: Efficient Camera Exposure Control for Visual Odometry via Deep Reinforcement Learning
• Authors: Shuyang Zhang, Jinhao He, Yilong Zhu, Jin Wu, Jie Yuan,
• Abstract summary: This study employs a deep reinforcement learning framework to train agents for exposure control. A lightweight image simulator is developed to facilitate the training process. Different levels of reward functions are crafted to enhance the VO systems.
• Score: 10.886819238167286
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The stability of visual odometry (VO) systems is undermined by degraded image quality, especially in environments with significant illumination changes. This study employs a deep reinforcement learning (DRL) framework to train agents for exposure control, aiming to enhance imaging performance in challenging conditions. A lightweight image simulator is developed to facilitate the training process, enabling the diversification of image exposure and sequence trajectory. This setup enables completely offline training, eliminating the need for direct interaction with camera hardware and the real environments. Different levels of reward functions are crafted to enhance the VO systems, equipping the DRL agents with varying intelligence. Extensive experiments have shown that our exposure control agents achieve superior efficiency-with an average inference duration of 1.58 ms per frame on a CPU-and respond more quickly than traditional feedback control schemes. By choosing an appropriate reward function, agents acquire an intelligent understanding of motion trends and anticipate future illumination changes. This predictive capability allows VO systems to deliver more stable and precise odometry results. The codes and datasets are available at https://github.com/ShuyangUni/drl_exposure_ctrl.

Related papers

• RoboKoop: Efficient Control Conditioned Representations from Visual Input in Robotics using Koopman Operator [14.77553682217217]
  We introduce a Contrastive Spectral Koopman Embedding network that allows us to learn efficient linearized visual representations from the agent's visual data in a high dimensional latent space. Our method enhances stability and control in gradient dynamics over time, significantly outperforming existing approaches.
  arXiv  Detail & Related papers  (2024-09-04T22:14:59Z)
• Light the Night: A Multi-Condition Diffusion Framework for Unpaired Low-Light Enhancement in Autonomous Driving [45.97279394690308]
  LightDiff is a framework designed to enhance the low-light image quality for autonomous driving applications. It incorporates a novel multi-condition adapter that adaptively controls the input weights from different modalities, including depth maps, RGB images, and text captions. It can significantly improve the performance of several state-of-the-art 3D detectors in night-time conditions while achieving high visual quality scores.
  arXiv  Detail & Related papers  (2024-04-07T04:10:06Z)
• Learning to Control Camera Exposure via Reinforcement Learning [8.359692000028891]
  Poorly adjusted camera exposure often leads to critical failure and performance degradation. Traditional camera exposure control methods require multiple convergence steps and time-consuming processes. We propose a new camera exposure control framework that rapidly controls camera exposure while performing real-time processing.
  arXiv  Detail & Related papers  (2024-04-02T04:53:39Z)
• A Non-Uniform Low-Light Image Enhancement Method with Multi-Scale Attention Transformer and Luminance Consistency Loss [11.585269110131659]
  Low-light image enhancement aims to improve the perception of images collected in dim environments. Existing methods cannot adaptively extract the differentiated luminance information, which will easily cause over-exposure and under-exposure. We propose a multi-scale attention Transformer named MSATr, which sufficiently extracts local and global features for light balance to improve the visual quality.
  arXiv  Detail & Related papers  (2023-12-27T10:07:11Z)
• Improving Lens Flare Removal with General Purpose Pipeline and Multiple Light Sources Recovery [69.71080926778413]
  flare artifacts can affect image visual quality and downstream computer vision tasks. Current methods do not consider automatic exposure and tone mapping in image signal processing pipeline. We propose a solution to improve the performance of lens flare removal by revisiting the ISP and design a more reliable light sources recovery strategy.
  arXiv  Detail & Related papers  (2023-08-31T04:58:17Z)
• Ultra-High-Definition Low-Light Image Enhancement: A Benchmark and Transformer-Based Method [51.30748775681917]
  We consider the task of low-light image enhancement (LLIE) and introduce a large-scale database consisting of images at 4K and 8K resolution. We conduct systematic benchmarking studies and provide a comparison of current LLIE algorithms. As a second contribution, we introduce LLFormer, a transformer-based low-light enhancement method.
  arXiv  Detail & Related papers  (2022-12-22T09:05:07Z)
• Towards Scale-Aware, Robust, and Generalizable Unsupervised Monocular Depth Estimation by Integrating IMU Motion Dynamics [74.1720528573331]
  Unsupervised monocular depth and ego-motion estimation has drawn extensive research attention in recent years. We propose DynaDepth, a novel scale-aware framework that integrates information from vision and IMU motion dynamics. We validate the effectiveness of DynaDepth by conducting extensive experiments and simulations on the KITTI and Make3D datasets.
  arXiv  Detail & Related papers  (2022-07-11T07:50:22Z)
• Toward Fast, Flexible, and Robust Low-Light Image Enhancement [87.27326390675155]
  We develop a new Self-Calibrated Illumination (SCI) learning framework for fast, flexible, and robust brightening images in real-world low-light scenarios. Considering the computational burden of the cascaded pattern, we construct the self-calibrated module which realizes the convergence between results of each stage. We make comprehensive explorations to SCI's inherent properties including operation-insensitive adaptability and model-irrelevant generality.
  arXiv  Detail & Related papers  (2022-04-21T14:40:32Z)
• Activation to Saliency: Forming High-Quality Labels for Unsupervised Salient Object Detection [54.92703325989853]
  We propose a two-stage Activation-to-Saliency (A2S) framework that effectively generates high-quality saliency cues. No human annotations are involved in our framework during the whole training process. Our framework reports significant performance compared with existing USOD methods.
  arXiv  Detail & Related papers  (2021-12-07T11:54:06Z)
• Learned Camera Gain and Exposure Control for Improved Visual Feature Detection and Matching [12.870196901446208]
  We explore a data-driven approach to account for environmental lighting changes, improving the quality of images for use in visual odometry (VO) or visual simultaneous localization and mapping (SLAM) We train a deep convolutional neural network model to predictively adjust camera gain and exposure time parameters. We demonstrate through extensive real-world experiments that our network can anticipate and compensate for dramatic lighting changes.
  arXiv  Detail & Related papers  (2021-02-08T16:46:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.