LW-ISP: A Lightweight Model with ISP and Deep Learning

• URL: http://arxiv.org/abs/2210.03904v1
• Date: Sat, 8 Oct 2022 04:00:03 GMT
• Title: LW-ISP: A Lightweight Model with ISP and Deep Learning
• Authors: Hongyang Chen and Kaisheng Ma
• Abstract summary: We show the possibility of learning-based method to achieve real-time high-performance processing in the ISP pipeline. We propose LW-ISP, a novel architecture designed to implicitly learn the image mapping from RAW data to RGB image. Experiments demonstrate that LW-ISP has achieved a 0.38 dB improvement in PSNR compared to the previous best method.
• Score: 17.972611191715888
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The deep learning (DL)-based methods of low-level tasks have many advantages over the traditional camera in terms of hardware prospects, error accumulation and imaging effects. Recently, the application of deep learning to replace the image signal processing (ISP) pipeline has appeared one after another; however, there is still a long way to go towards real landing. In this paper, we show the possibility of learning-based method to achieve real-time high-performance processing in the ISP pipeline. We propose LW-ISP, a novel architecture designed to implicitly learn the image mapping from RAW data to RGB image. Based on U-Net architecture, we propose the fine-grained attention module and a plug-and-play upsampling block suitable for low-level tasks. In particular, we design a heterogeneous distillation algorithm to distill the implicit features and reconstruction information of the clean image, so as to guide the learning of the student model. Our experiments demonstrate that LW-ISP has achieved a 0.38 dB improvement in PSNR compared to the previous best method, while the model parameters and calculation have been reduced by 23 times and 81 times. The inference efficiency has been accelerated by at least 15 times. Without bells and whistles, LW-ISP has achieved quite competitive results in ISP subtasks including image denoising and enhancement.

Related papers

• Towards Realistic Low-Light Image Enhancement via ISP Driven Data Modeling [61.95831392879045]
  Deep neural networks (DNNs) have recently become the leading method for low-light image enhancement (LLIE) Despite significant progress, their outputs may still exhibit issues such as amplified noise, incorrect white balance, or unnatural enhancements when deployed in real world applications. A key challenge is the lack of diverse, large scale training data that captures the complexities of low-light conditions and imaging pipelines. We propose a novel image signal processing (ISP) driven data synthesis pipeline that addresses these challenges by generating unlimited paired training data.
  arXiv  Detail & Related papers  (2025-04-16T15:53:53Z)
• Striving for Faster and Better: A One-Layer Architecture with Auto Re-parameterization for Low-Light Image Enhancement [50.93686436282772]
  We aim to delve into the limits of image enhancers both from visual quality and computational efficiency. By rethinking the task demands, we build an explicit connection, i.e., visual quality and computational efficiency are corresponding to model learning and structure design. Ultimately, this achieves efficient low-light image enhancement using only a single convolutional layer, while maintaining excellent visual quality.
  arXiv  Detail & Related papers  (2025-02-27T08:20:03Z)
• DGNet: Dynamic Gradient-Guided Network for Water-Related Optics Image Enhancement [77.0360085530701]
  Underwater image enhancement (UIE) is a challenging task due to the complex degradation caused by underwater environments. Previous methods often idealize the degradation process, and neglect the impact of medium noise and object motion on the distribution of image features. Our approach utilizes predicted images to dynamically update pseudo-labels, adding a dynamic gradient to optimize the network's gradient space.
  arXiv  Detail & Related papers  (2023-12-12T06:07:21Z)
• LDM-ISP: Enhancing Neural ISP for Low Light with Latent Diffusion Models [54.93010869546011]
  We propose to leverage the pre-trained latent diffusion model to perform the neural ISP for enhancing extremely low-light images. Specifically, to tailor the pre-trained latent diffusion model to operate on the RAW domain, we train a set of lightweight taming modules. We observe different roles of UNet denoising and decoder reconstruction in the latent diffusion model, which inspires us to decompose the low-light image enhancement task into latent-space low-frequency content generation and decoding-phase high-frequency detail maintenance.
  arXiv  Detail & Related papers  (2023-12-02T04:31:51Z)
• Learning Degradation-Independent Representations for Camera ISP Pipelines [14.195578257521934]
  We propose a novel approach to learn degradation-independent representations (DiR) through the refinement of a self-supervised learned baseline representation. The proposed DiR learning technique has remarkable domain generalization capability and it outperforms state-of-the-art methods across various downstream tasks.
  arXiv  Detail & Related papers  (2023-07-03T05:38:28Z)
• Ultra-High-Definition Low-Light Image Enhancement: A Benchmark and Transformer-Based Method [51.30748775681917]
  We consider the task of low-light image enhancement (LLIE) and introduce a large-scale database consisting of images at 4K and 8K resolution. We conduct systematic benchmarking studies and provide a comparison of current LLIE algorithms. As a second contribution, we introduce LLFormer, a transformer-based low-light enhancement method.
  arXiv  Detail & Related papers  (2022-12-22T09:05:07Z)
• Combining Attention Module and Pixel Shuffle for License Plate Super-Resolution [3.8831062015253055]
  This work focuses on license plate (LP) reconstruction in low-resolution and low-quality images. We present a Single-Image Super-Resolution (SISR) approach that extends the attention/transformer module concept. In our experiments, the proposed method outperformed the baselines both quantitatively and qualitatively.
  arXiv  Detail & Related papers  (2022-10-30T13:05:07Z)
• Enabling ISP-less Low-Power Computer Vision [4.102254385058941]
  We release the raw version of a large-scale benchmark for generic high-level vision tasks. For ISP-less CV systems, training on raw images result in a 7.1% increase in test accuracy. We propose an energy-efficient form of analog in-pixel demosaicing that may be coupled with in-pixel CNN computations.
  arXiv  Detail & Related papers  (2022-10-11T13:47:30Z)
• Learning Deep Context-Sensitive Decomposition for Low-Light Image Enhancement [58.72667941107544]
  A typical framework is to simultaneously estimate the illumination and reflectance, but they disregard the scene-level contextual information encapsulated in feature spaces. We develop a new context-sensitive decomposition network architecture to exploit the scene-level contextual dependencies on spatial scales. We develop a lightweight CSDNet (named LiteCSDNet) by reducing the number of channels.
  arXiv  Detail & Related papers  (2021-12-09T06:25:30Z)
• SPI-GAN: Towards Single-Pixel Imaging through Generative Adversarial Network [6.722629246312285]
  We propose a generative adversarial network-based reconstruction framework for single-pixel imaging, referred to as SPI-GAN. Our method can reconstruct images with 17.92 dB PSNR and 0.487 SSIM, even if the sampling ratio drops to 5%.
  arXiv  Detail & Related papers  (2021-07-03T03:06:09Z)
• Pseudo-ISP: Learning Pseudo In-camera Signal Processing Pipeline from A Color Image Denoiser [73.88164217509816]
  We present an unpaired learning scheme to adapt a color image denoiser for handling test images with noise discrepancy. We consider a practical training setting, i.e., a pre-trained denoiser, a set of test noisy images, and an unpaired set of clean images. Pseudo-ISP is effective in synthesizing realistic noisy sRGB images.
  arXiv  Detail & Related papers  (2021-03-18T13:11:28Z)
• BP-DIP: A Backprojection based Deep Image Prior [49.375539602228415]
  We propose two image restoration approaches: (i) Deep Image Prior (DIP), which trains a convolutional neural network (CNN) from scratch in test time using the degraded image; and (ii) a backprojection (BP) fidelity term, which is an alternative to the standard least squares loss that is usually used in previous DIP works. We demonstrate the performance of the proposed method, termed BP-DIP, on the deblurring task and show its advantages over the plain DIP, with both higher PSNR values and better inference run-time.
  arXiv  Detail & Related papers  (2020-03-11T17:09:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.