Del-Net: A Single-Stage Network for Mobile Camera ISP

• URL: http://arxiv.org/abs/2108.01623v1
• Date: Tue, 3 Aug 2021 16:51:11 GMT
• Title: Del-Net: A Single-Stage Network for Mobile Camera ISP
• Authors: Saumya Gupta, Diplav Srivastava, Umang Chaturvedi, Anurag Jain, Gaurav Khandelwal
• Abstract summary: Traditional image signal processing (ISP) pipeline in a smartphone camera consists of several image processing steps performed sequentially to reconstruct a high quality sRGB image from the raw sensor data. Deep learning methods using convolutional neural networks (CNN) have become popular in solving many image-related tasks such as image denoising, contrast enhancement, super resolution, deblurring, etc. In this paper we propose DelNet - a single end-to-end deep learning model - to learn the entire ISP pipeline within reasonable complexity for smartphone deployment.
• Score: 14.168130234198467
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The quality of images captured by smartphones is an important specification since smartphones are becoming ubiquitous as primary capturing devices. The traditional image signal processing (ISP) pipeline in a smartphone camera consists of several image processing steps performed sequentially to reconstruct a high quality sRGB image from the raw sensor data. These steps consist of demosaicing, denoising, white balancing, gamma correction, colour enhancement, etc. Since each of them are performed sequentially using hand-crafted algorithms, the residual error from each processing module accumulates in the final reconstructed signal. Thus, the traditional ISP pipeline has limited reconstruction quality in terms of generalizability across different lighting conditions and associated noise levels while capturing the image. Deep learning methods using convolutional neural networks (CNN) have become popular in solving many image-related tasks such as image denoising, contrast enhancement, super resolution, deblurring, etc. Furthermore, recent approaches for the RAW to sRGB conversion using deep learning methods have also been published, however, their immense complexity in terms of their memory requirement and number of Mult-Adds make them unsuitable for mobile camera ISP. In this paper we propose DelNet - a single end-to-end deep learning model - to learn the entire ISP pipeline within reasonable complexity for smartphone deployment. Del-Net is a multi-scale architecture that uses spatial and channel attention to capture global features like colour, as well as a series of lightweight modified residual attention blocks to help with denoising. For validation, we provide results to show the proposed Del-Net achieves compelling reconstruction quality.

Related papers

• Simple Image Signal Processing using Global Context Guidance [56.41827271721955]
  Deep learning-based ISPs aim to transform RAW images into DSLR-like RGB images using deep neural networks. We propose a novel module that can be integrated into any neural ISP to capture the global context information from the full RAW images. Our model achieves state-of-the-art results on different benchmarks using diverse and real smartphone images.
  arXiv  Detail & Related papers  (2024-04-17T17:11:47Z)
• Rawformer: Unpaired Raw-to-Raw Translation for Learnable Camera ISPs [53.68932498994655]
  This paper introduces a novel method for unpaired learning of raw-to-raw translation across diverse cameras. It accurately maps raw images captured by a certain camera to the target camera, facilitating the generalization of learnable ISPs to new unseen cameras. Our method demonstrates superior performance on real camera datasets, achieving higher accuracy compared to previous state-of-the-art techniques.
  arXiv  Detail & Related papers  (2024-04-16T16:17:48Z)
• Learning Degradation-Independent Representations for Camera ISP Pipelines [14.195578257521934]
  We propose a novel approach to learn degradation-independent representations (DiR) through the refinement of a self-supervised learned baseline representation. The proposed DiR learning technique has remarkable domain generalization capability and it outperforms state-of-the-art methods across various downstream tasks.
  arXiv  Detail & Related papers  (2023-07-03T05:38:28Z)
• Learning Enriched Features for Fast Image Restoration and Enhancement [166.17296369600774]
  This paper presents a holistic goal of maintaining spatially-precise high-resolution representations through the entire network. We learn an enriched set of features that combines contextual information from multiple scales, while simultaneously preserving the high-resolution spatial details. Our approach achieves state-of-the-art results for a variety of image processing tasks, including defocus deblurring, image denoising, super-resolution, and image enhancement.
  arXiv  Detail & Related papers  (2022-04-19T17:59:45Z)
• RBSRICNN: Raw Burst Super-Resolution through Iterative Convolutional Neural Network [23.451063587138393]
  We propose a Raw Burst Super-Resolution Iterative Convolutional Neural Network (RBSRICNN) The proposed network produces the final output by an iterative refinement of the intermediate SR estimates. We demonstrate the effectiveness of our proposed approach in quantitative and qualitative experiments.
  arXiv  Detail & Related papers  (2021-10-25T19:01:28Z)
• PyNET-CA: Enhanced PyNET with Channel Attention for End-to-End Mobile Image Signal Processing [32.7355302269855]
  We propose PyNET-CA, an end-to-end mobile ISP deep learning algorithm for RAW to RGB reconstruction. We demonstrate the performance of the proposed method with comparative experiments and results from the AIM 2020 learned smartphone ISP challenge.
  arXiv  Detail & Related papers  (2021-04-07T03:40:11Z)
• Deep Burst Super-Resolution [165.90445859851448]
  We propose a novel architecture for the burst super-resolution task. Our network takes multiple noisy RAW images as input, and generates a denoised, super-resolved RGB image as output. In order to enable training and evaluation on real-world data, we additionally introduce the BurstSR dataset.
  arXiv  Detail & Related papers  (2021-01-26T18:57:21Z)
• AWNet: Attentive Wavelet Network for Image ISP [14.58067200317891]
  We introduce a novel network that utilizes the attention mechanism and wavelet transform, dubbed AWNet, to tackle this learnable image ISP problem. Our proposed method enables us to restore favorable image details from RAW information and achieve a larger receptive field. Experimental results indicate the advances of our design in both qualitative and quantitative measurements.
  arXiv  Detail & Related papers  (2020-08-20T23:28:41Z)
• EventSR: From Asynchronous Events to Image Reconstruction, Restoration, and Super-Resolution via End-to-End Adversarial Learning [75.17497166510083]
  Event cameras sense intensity changes and have many advantages over conventional cameras. Some methods have been proposed to reconstruct intensity images from event streams. The outputs are still in low resolution (LR), noisy, and unrealistic. We propose a novel end-to-end pipeline that reconstructs LR images from event streams, enhances the image qualities and upsamples the enhanced images, called EventSR.
  arXiv  Detail & Related papers  (2020-03-17T10:58:10Z)
• Learning Enriched Features for Real Image Restoration and Enhancement [166.17296369600774]
  convolutional neural networks (CNNs) have achieved dramatic improvements over conventional approaches for image restoration task. We present a novel architecture with the collective goals of maintaining spatially-precise high-resolution representations through the entire network. Our approach learns an enriched set of features that combines contextual information from multiple scales, while simultaneously preserving the high-resolution spatial details.
  arXiv  Detail & Related papers  (2020-03-15T11:04:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.