Light the Night: A Multi-Condition Diffusion Framework for Unpaired Low-Light Enhancement in Autonomous Driving

• URL: http://arxiv.org/abs/2404.04804v1
• Date: Sun, 7 Apr 2024 04:10:06 GMT
• Title: Light the Night: A Multi-Condition Diffusion Framework for Unpaired Low-Light Enhancement in Autonomous Driving
• Authors: Jinlong Li, Baolu Li, Zhengzhong Tu, Xinyu Liu, Qing Guo, Felix Juefei-Xu, Runsheng Xu, Hongkai Yu,
• Abstract summary: LightDiff is a framework designed to enhance the low-light image quality for autonomous driving applications. It incorporates a novel multi-condition adapter that adaptively controls the input weights from different modalities, including depth maps, RGB images, and text captions. It can significantly improve the performance of several state-of-the-art 3D detectors in night-time conditions while achieving high visual quality scores.
• Score: 45.97279394690308
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision-centric perception systems for autonomous driving have gained considerable attention recently due to their cost-effectiveness and scalability, especially compared to LiDAR-based systems. However, these systems often struggle in low-light conditions, potentially compromising their performance and safety. To address this, our paper introduces LightDiff, a domain-tailored framework designed to enhance the low-light image quality for autonomous driving applications. Specifically, we employ a multi-condition controlled diffusion model. LightDiff works without any human-collected paired data, leveraging a dynamic data degradation process instead. It incorporates a novel multi-condition adapter that adaptively controls the input weights from different modalities, including depth maps, RGB images, and text captions, to effectively illuminate dark scenes while maintaining context consistency. Furthermore, to align the enhanced images with the detection model's knowledge, LightDiff employs perception-specific scores as rewards to guide the diffusion training process through reinforcement learning. Extensive experiments on the nuScenes datasets demonstrate that LightDiff can significantly improve the performance of several state-of-the-art 3D detectors in night-time conditions while achieving high visual quality scores, highlighting its potential to safeguard autonomous driving.

Related papers

• CodeEnhance: A Codebook-Driven Approach for Low-Light Image Enhancement [97.95330185793358]
  Low-light image enhancement (LLIE) aims to improve low-illumination images. Existing methods face two challenges: uncertainty in restoration from diverse brightness degradations and loss of texture and color information. We propose a novel enhancement approach, CodeEnhance, by leveraging quantized priors and image refinement.
  arXiv  Detail & Related papers  (2024-04-08T07:34:39Z)
• Zero-Reference Low-Light Enhancement via Physical Quadruple Priors [58.77377454210244]
  We propose a new zero-reference low-light enhancement framework trainable solely with normal light images. This framework is able to restore our illumination-invariant prior back to images, automatically achieving low-light enhancement.
  arXiv  Detail & Related papers  (2024-03-19T17:36:28Z)
• NiteDR: Nighttime Image De-Raining with Cross-View Sensor Cooperative Learning for Dynamic Driving Scenes [49.92839157944134]
  In nighttime driving scenes, insufficient and uneven lighting shrouds the scenes in darkness, resulting degradation of image quality and visibility. We develop an image de-raining framework tailored for rainy nighttime driving scenes. It aims to remove rain artifacts, enrich scene representation, and restore useful information.
  arXiv  Detail & Related papers  (2024-02-28T09:02:33Z)
• A Non-Uniform Low-Light Image Enhancement Method with Multi-Scale Attention Transformer and Luminance Consistency Loss [11.585269110131659]
  Low-light image enhancement aims to improve the perception of images collected in dim environments. Existing methods cannot adaptively extract the differentiated luminance information, which will easily cause over-exposure and under-exposure. We propose a multi-scale attention Transformer named MSATr, which sufficiently extracts local and global features for light balance to improve the visual quality.
  arXiv  Detail & Related papers  (2023-12-27T10:07:11Z)
• CLE Diffusion: Controllable Light Enhancement Diffusion Model [80.62384873945197]
  Controllable Light Enhancement Diffusion Model, dubbed CLE Diffusion, is a novel diffusion framework to provide users with rich controllability. Built with a conditional diffusion model, we introduce an illumination embedding to let users control their desired brightness level.
  arXiv  Detail & Related papers  (2023-08-13T09:05:56Z)
• Diffusion in the Dark: A Diffusion Model for Low-Light Text Recognition [78.50328335703914]
  Diffusion in the Dark (DiD) is a diffusion model for low-light image reconstruction for text recognition. We demonstrate that DiD, without any task-specific optimization, can outperform SOTA low-light methods in low-light text recognition on real images.
  arXiv  Detail & Related papers  (2023-03-07T23:52:51Z)
• Does Thermal data make the detection systems more reliable? [1.2891210250935146]
  We propose a comprehensive detection system based on a multimodal-collaborative framework. This framework learns from both RGB (from visual cameras) and thermal (from Infrared cameras) data. Our empirical results show that while the improvement in accuracy is nominal, the value lies in challenging and extremely difficult edge cases.
  arXiv  Detail & Related papers  (2021-11-09T15:04:34Z)
• Bridge the Vision Gap from Field to Command: A Deep Learning Network Enhancing Illumination and Details [17.25188250076639]
  We propose a two-stream framework named NEID to tune up the brightness and enhance the details simultaneously. The proposed method consists of three parts: Light Enhancement (LE), Detail Refinement (DR) and Feature Fusing (FF) module.
  arXiv  Detail & Related papers  (2021-01-20T09:39:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.