Instance Segmentation in the Dark

• URL: http://arxiv.org/abs/2304.14298v2
• Date: Fri, 8 Sep 2023 18:45:06 GMT
• Title: Instance Segmentation in the Dark
• Authors: Linwei Chen, Ying Fu, Kaixuan Wei, Dezhi Zheng, Felix Heide
• Abstract summary: We take a deep look at instance segmentation in the dark and introduce several techniques that substantially boost the low-light inference accuracy. We propose a novel learning method that relies on an adaptive weighted downsampling layer, a smooth-oriented convolutional block, and disturbance suppression learning. We capture a real-world low-light instance segmentation dataset comprising over two thousand paired low/normal-light images with instance-level pixel-wise annotations.
• Score: 43.85818645776587
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Existing instance segmentation techniques are primarily tailored for high-visibility inputs, but their performance significantly deteriorates in extremely low-light environments. In this work, we take a deep look at instance segmentation in the dark and introduce several techniques that substantially boost the low-light inference accuracy. The proposed method is motivated by the observation that noise in low-light images introduces high-frequency disturbances to the feature maps of neural networks, thereby significantly degrading performance. To suppress this ``feature noise", we propose a novel learning method that relies on an adaptive weighted downsampling layer, a smooth-oriented convolutional block, and disturbance suppression learning. These components effectively reduce feature noise during downsampling and convolution operations, enabling the model to learn disturbance-invariant features. Furthermore, we discover that high-bit-depth RAW images can better preserve richer scene information in low-light conditions compared to typical camera sRGB outputs, thus supporting the use of RAW-input algorithms. Our analysis indicates that high bit-depth can be critical for low-light instance segmentation. To mitigate the scarcity of annotated RAW datasets, we leverage a low-light RAW synthetic pipeline to generate realistic low-light data. In addition, to facilitate further research in this direction, we capture a real-world low-light instance segmentation dataset comprising over two thousand paired low/normal-light images with instance-level pixel-wise annotations. Remarkably, without any image preprocessing, we achieve satisfactory performance on instance segmentation in very low light (4~\% AP higher than state-of-the-art competitors), meanwhile opening new opportunities for future research.

Related papers

• Multi-Scale Denoising in the Feature Space for Low-Light Instance Segmentation [2.642212767247493]
  Instance segmentation for low-light imagery remains largely unexplored. Our proposed method implements weighted non-local blocks (wNLB) in the feature extractor. We introduce additional learnable weights at each layer in order to enhance the network's adaptability to real-world noise characteristics.
  arXiv  Detail & Related papers  (2024-02-28T13:07:16Z)
• LDM-ISP: Enhancing Neural ISP for Low Light with Latent Diffusion Models [54.93010869546011]
  We propose to leverage the pre-trained latent diffusion model to perform the neural ISP for enhancing extremely low-light images. Specifically, to tailor the pre-trained latent diffusion model to operate on the RAW domain, we train a set of lightweight taming modules. We observe different roles of UNet denoising and decoder reconstruction in the latent diffusion model, which inspires us to decompose the low-light image enhancement task into latent-space low-frequency content generation and decoding-phase high-frequency detail maintenance.
  arXiv  Detail & Related papers  (2023-12-02T04:31:51Z)
• Advancing Unsupervised Low-light Image Enhancement: Noise Estimation, Illumination Interpolation, and Self-Regulation [55.07472635587852]
  Low-Light Image Enhancement (LLIE) techniques have made notable advancements in preserving image details and enhancing contrast. These approaches encounter persistent challenges in efficiently mitigating dynamic noise and accommodating diverse low-light scenarios. We first propose a method for estimating the noise level in low light images in a quick and accurate way. We then devise a Learnable Illumination Interpolator (LII) to satisfy general constraints between illumination and input.
  arXiv  Detail & Related papers  (2023-05-17T13:56:48Z)
• Seeing Through The Noisy Dark: Toward Real-world Low-Light Image Enhancement and Denoising [125.56062454927755]
  Real-world low-light environment usually suffer from lower visibility and heavier noise, due to insufficient light or hardware limitation. We propose a novel end-to-end method termed Real-world Low-light Enhancement & Denoising Network (RLED-Net)
  arXiv  Detail & Related papers  (2022-10-02T14:57:23Z)
• KinD-LCE Curve Estimation And Retinex Fusion On Low-Light Image [7.280719886684936]
  This paper proposes an algorithm for low illumination enhancement. KinD-LCE uses a light curve estimation module to enhance the illumination map in the Retinex decomposed image. An illumination map and reflection map fusion module were also proposed to restore the image details and reduce detail loss.
  arXiv  Detail & Related papers  (2022-07-19T11:49:21Z)
• Adaptive Unfolding Total Variation Network for Low-Light Image Enhancement [6.531546527140475]
  Most existing enhancing algorithms in sRGB space only focus on the low visibility problem or suppress the noise under a hypothetical noise level. We propose an adaptive unfolding total variation network (UTVNet) to approximate the noise level from the real sRGB low-light image. Experiments on real-world low-light images clearly demonstrate the superior performance of UTVNet over state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-10-03T11:22:17Z)
• Deep Bilateral Retinex for Low-Light Image Enhancement [96.15991198417552]
  Low-light images suffer from poor visibility caused by low contrast, color distortion and measurement noise. This paper proposes a deep learning method for low-light image enhancement with a particular focus on handling the measurement noise. The proposed method is very competitive to the state-of-the-art methods, and has significant advantage over others when processing images captured in extremely low lighting conditions.
  arXiv  Detail & Related papers  (2020-07-04T06:26:44Z)
• Unsupervised Low-light Image Enhancement with Decoupled Networks [103.74355338972123]
  We learn a two-stage GAN-based framework to enhance the real-world low-light images in a fully unsupervised fashion. Our proposed method outperforms the state-of-the-art unsupervised image enhancement methods in terms of both illumination enhancement and noise reduction.
  arXiv  Detail & Related papers  (2020-05-06T13:37:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.