Unrolled Decomposed Unpaired Learning for Controllable Low-Light Video Enhancement

• URL: http://arxiv.org/abs/2408.12316v1
• Date: Thu, 22 Aug 2024 11:45:11 GMT
• Title: Unrolled Decomposed Unpaired Learning for Controllable Low-Light Video Enhancement
• Authors: Lingyu Zhu, Wenhan Yang, Baoliang Chen, Hanwei Zhu, Zhangkai Ni, Qi Mao, Shiqi Wang,
• Abstract summary: This paper makes endeavors in the direction of learning for low-light video enhancement without using paired ground truth. Compared to low-light image enhancement, enhancing low-light videos is more difficult due to the intertwined effects of noise, exposure, and contrast in the spatial domain, jointly with the need for temporal coherence. We propose the Unrolled Decomposed Unpaired Network (UDU-Net) for enhancing low-light videos by unrolling the optimization functions into a deep network to decompose the signal into spatial and temporal-related factors, which are updated iteratively.
• Score: 48.76608212565327
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Obtaining pairs of low/normal-light videos, with motions, is more challenging than still images, which raises technical issues and poses the technical route of unpaired learning as a critical role. This paper makes endeavors in the direction of learning for low-light video enhancement without using paired ground truth. Compared to low-light image enhancement, enhancing low-light videos is more difficult due to the intertwined effects of noise, exposure, and contrast in the spatial domain, jointly with the need for temporal coherence. To address the above challenge, we propose the Unrolled Decomposed Unpaired Network (UDU-Net) for enhancing low-light videos by unrolling the optimization functions into a deep network to decompose the signal into spatial and temporal-related factors, which are updated iteratively. Firstly, we formulate low-light video enhancement as a Maximum A Posteriori estimation (MAP) problem with carefully designed spatial and temporal visual regularization. Then, via unrolling the problem, the optimization of the spatial and temporal constraints can be decomposed into different steps and updated in a stage-wise manner. From the spatial perspective, the designed Intra subnet leverages unpair prior information from expert photography retouched skills to adjust the statistical distribution. Additionally, we introduce a novel mechanism that integrates human perception feedback to guide network optimization, suppressing over/under-exposure conditions. Meanwhile, to address the issue from the temporal perspective, the designed Inter subnet fully exploits temporal cues in progressive optimization, which helps achieve improved temporal consistency in enhancement results. Consequently, the proposed method achieves superior performance to state-of-the-art methods in video illumination, noise suppression, and temporal consistency across outdoor and indoor scenes.

Related papers

• Towards Real-world Event-guided Low-light Video Enhancement and Deblurring [39.942568142125126]
  Event cameras have emerged as a promising solution for improving image quality in low-light environments. We introduce an end-to-end framework to effectively handle these tasks. Our framework incorporates a module to efficiently leverage temporal information from events and frames.
  arXiv  Detail & Related papers  (2024-08-27T09:44:54Z)
• Low-Light Video Enhancement via Spatial-Temporal Consistent Illumination and Reflection Decomposition [68.6707284662443]
  Low-Light Video Enhancement (LLVE) seeks to restore dynamic and static scenes plagued by severe invisibility and noise. One critical aspect is formulating a consistency constraint specifically for temporal-spatial illumination and appearance enhanced versions. We present an innovative video Retinex-based decomposition strategy that operates without the need for explicit supervision.
  arXiv  Detail & Related papers  (2024-05-24T15:56:40Z)
• FRESCO: Spatial-Temporal Correspondence for Zero-Shot Video Translation [85.29772293776395]
  We introduce FRESCO, intra-frame correspondence alongside inter-frame correspondence to establish a more robust spatial-temporal constraint. This enhancement ensures a more consistent transformation of semantically similar content across frames. Our approach involves an explicit update of features to achieve high spatial-temporal consistency with the input video.
  arXiv  Detail & Related papers  (2024-03-19T17:59:18Z)
• Low-light Image and Video Enhancement via Selective Manipulation of Chromaticity [1.4680035572775534]
  We present a simple yet effective approach for low-light image and video enhancement. The above adaptivity allows us to avoid the costly step of low-light image decomposition into illumination and reflectance. Our results on standard lowlight image datasets show the efficacy of our algorithm and its qualitative and quantitative superiority over several state-of-the-art techniques.
  arXiv  Detail & Related papers  (2022-03-09T17:01:28Z)
• Invertible Network for Unpaired Low-light Image Enhancement [78.33382003460903]
  We propose to leverage the invertible network to enhance low-light image in forward process and degrade the normal-light one inversely with unpaired learning. In addition to the adversarial loss, we design various loss functions to ensure the stability of training and preserve more image details. We present a progressive self-guided enhancement process for low-light images and achieve favorable performance against the SOTAs.
  arXiv  Detail & Related papers  (2021-12-24T17:00:54Z)
• Video Salient Object Detection via Contrastive Features and Attention Modules [106.33219760012048]
  We propose a network with attention modules to learn contrastive features for video salient object detection. A co-attention formulation is utilized to combine the low-level and high-level features. We show that the proposed method requires less computation, and performs favorably against the state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-11-03T17:40:32Z)
• Progressive Joint Low-light Enhancement and Noise Removal for Raw Images [10.778200442212334]
  Low-light imaging on mobile devices is typically challenging due to insufficient incident light coming through the relatively small aperture. We propose a low-light image processing framework that performs joint illumination adjustment, color enhancement, and denoising. Our framework does not need to recollect massive data when being adapted to another camera model.
  arXiv  Detail & Related papers  (2021-06-28T16:43:52Z)
• Enhanced Spatio-Temporal Interaction Learning for Video Deraining: A Faster and Better Framework [93.37833982180538]
  Video deraining is an important task in computer vision as the unwanted rain hampers the visibility of videos and deteriorates the robustness of most outdoor vision systems. We present a new end-to-end deraining framework, named Enhanced Spatio-Temporal Interaction Network (ESTINet) ESTINet considerably boosts current state-of-the-art video deraining quality and speed.
  arXiv  Detail & Related papers  (2021-03-23T05:19:35Z)
• Contextual colorization and denoising for low-light ultra high resolution sequences [0.0]
  Low-light image sequences generally suffer from incoherent noise, flicker and blurring of objects and moving objects. We tackle these problems with an unpaired-learning method that offers simultaneous colorization and denoising. We show that our method outperforms existing approaches in terms of subjective quality and that it is robust to variations in brightness levels and noise.
  arXiv  Detail & Related papers  (2021-01-05T15:35:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.