LW-ISP: A Lightweight Model with ISP and Deep Learning
- URL: http://arxiv.org/abs/2210.03904v1
- Date: Sat, 8 Oct 2022 04:00:03 GMT
- Title: LW-ISP: A Lightweight Model with ISP and Deep Learning
- Authors: Hongyang Chen and Kaisheng Ma
- Abstract summary: We show the possibility of learning-based method to achieve real-time high-performance processing in the ISP pipeline.
We propose LW-ISP, a novel architecture designed to implicitly learn the image mapping from RAW data to RGB image.
Experiments demonstrate that LW-ISP has achieved a 0.38 dB improvement in PSNR compared to the previous best method.
- Score: 17.972611191715888
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The deep learning (DL)-based methods of low-level tasks have many advantages
over the traditional camera in terms of hardware prospects, error accumulation
and imaging effects. Recently, the application of deep learning to replace the
image signal processing (ISP) pipeline has appeared one after another; however,
there is still a long way to go towards real landing. In this paper, we show
the possibility of learning-based method to achieve real-time high-performance
processing in the ISP pipeline. We propose LW-ISP, a novel architecture
designed to implicitly learn the image mapping from RAW data to RGB image.
Based on U-Net architecture, we propose the fine-grained attention module and a
plug-and-play upsampling block suitable for low-level tasks. In particular, we
design a heterogeneous distillation algorithm to distill the implicit features
and reconstruction information of the clean image, so as to guide the learning
of the student model. Our experiments demonstrate that LW-ISP has achieved a
0.38 dB improvement in PSNR compared to the previous best method, while the
model parameters and calculation have been reduced by 23 times and 81 times.
The inference efficiency has been accelerated by at least 15 times. Without
bells and whistles, LW-ISP has achieved quite competitive results in ISP
subtasks including image denoising and enhancement.
Related papers
- DGNet: Dynamic Gradient-Guided Network for Water-Related Optics Image
Enhancement [77.0360085530701]
Underwater image enhancement (UIE) is a challenging task due to the complex degradation caused by underwater environments.
Previous methods often idealize the degradation process, and neglect the impact of medium noise and object motion on the distribution of image features.
Our approach utilizes predicted images to dynamically update pseudo-labels, adding a dynamic gradient to optimize the network's gradient space.
arXiv Detail & Related papers (2023-12-12T06:07:21Z) - LDM-ISP: Enhancing Neural ISP for Low Light with Latent Diffusion Models [54.93010869546011]
We propose to leverage the pre-trained latent diffusion model to perform the neural ISP for enhancing extremely low-light images.
Specifically, to tailor the pre-trained latent diffusion model to operate on the RAW domain, we train a set of lightweight taming modules.
We observe different roles of UNet denoising and decoder reconstruction in the latent diffusion model, which inspires us to decompose the low-light image enhancement task into latent-space low-frequency content generation and decoding-phase high-frequency detail maintenance.
arXiv Detail & Related papers (2023-12-02T04:31:51Z) - Learning Degradation-Independent Representations for Camera ISP Pipelines [14.195578257521934]
We propose a novel approach to learn degradation-independent representations (DiR) through the refinement of a self-supervised learned baseline representation.
The proposed DiR learning technique has remarkable domain generalization capability and it outperforms state-of-the-art methods across various downstream tasks.
arXiv Detail & Related papers (2023-07-03T05:38:28Z) - Ultra-High-Definition Low-Light Image Enhancement: A Benchmark and
Transformer-Based Method [51.30748775681917]
We consider the task of low-light image enhancement (LLIE) and introduce a large-scale database consisting of images at 4K and 8K resolution.
We conduct systematic benchmarking studies and provide a comparison of current LLIE algorithms.
As a second contribution, we introduce LLFormer, a transformer-based low-light enhancement method.
arXiv Detail & Related papers (2022-12-22T09:05:07Z) - Combining Attention Module and Pixel Shuffle for License Plate
Super-Resolution [3.8831062015253055]
This work focuses on license plate (LP) reconstruction in low-resolution and low-quality images.
We present a Single-Image Super-Resolution (SISR) approach that extends the attention/transformer module concept.
In our experiments, the proposed method outperformed the baselines both quantitatively and qualitatively.
arXiv Detail & Related papers (2022-10-30T13:05:07Z) - Enabling ISP-less Low-Power Computer Vision [4.102254385058941]
We release the raw version of a large-scale benchmark for generic high-level vision tasks.
For ISP-less CV systems, training on raw images result in a 7.1% increase in test accuracy.
We propose an energy-efficient form of analog in-pixel demosaicing that may be coupled with in-pixel CNN computations.
arXiv Detail & Related papers (2022-10-11T13:47:30Z) - Learning Deep Context-Sensitive Decomposition for Low-Light Image
Enhancement [58.72667941107544]
A typical framework is to simultaneously estimate the illumination and reflectance, but they disregard the scene-level contextual information encapsulated in feature spaces.
We develop a new context-sensitive decomposition network architecture to exploit the scene-level contextual dependencies on spatial scales.
We develop a lightweight CSDNet (named LiteCSDNet) by reducing the number of channels.
arXiv Detail & Related papers (2021-12-09T06:25:30Z) - SPI-GAN: Towards Single-Pixel Imaging through Generative Adversarial
Network [6.722629246312285]
We propose a generative adversarial network-based reconstruction framework for single-pixel imaging, referred to as SPI-GAN.
Our method can reconstruct images with 17.92 dB PSNR and 0.487 SSIM, even if the sampling ratio drops to 5%.
arXiv Detail & Related papers (2021-07-03T03:06:09Z) - Pseudo-ISP: Learning Pseudo In-camera Signal Processing Pipeline from A
Color Image Denoiser [73.88164217509816]
We present an unpaired learning scheme to adapt a color image denoiser for handling test images with noise discrepancy.
We consider a practical training setting, i.e., a pre-trained denoiser, a set of test noisy images, and an unpaired set of clean images.
Pseudo-ISP is effective in synthesizing realistic noisy sRGB images.
arXiv Detail & Related papers (2021-03-18T13:11:28Z) - BP-DIP: A Backprojection based Deep Image Prior [49.375539602228415]
We propose two image restoration approaches: (i) Deep Image Prior (DIP), which trains a convolutional neural network (CNN) from scratch in test time using the degraded image; and (ii) a backprojection (BP) fidelity term, which is an alternative to the standard least squares loss that is usually used in previous DIP works.
We demonstrate the performance of the proposed method, termed BP-DIP, on the deblurring task and show its advantages over the plain DIP, with both higher PSNR values and better inference run-time.
arXiv Detail & Related papers (2020-03-11T17:09:12Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.