ISP meets Deep Learning: A Survey on Deep Learning Methods for Image Signal Processing

• URL: http://arxiv.org/abs/2305.11994v2
• Date: Tue, 23 May 2023 12:17:39 GMT
• Title: ISP meets Deep Learning: A Survey on Deep Learning Methods for Image Signal Processing
• Authors: Matheus Henrique Marques da Silva, Jhessica Victoria Santos da Silva, Rodrigo Reis Arrais, Wladimir Barroso Guedes de Ara\'ujo Neto, Leonardo Tadeu Lopes, Guilherme Augusto Bileki, Iago Oliveira Lima, Lucas Borges Rondon, Bruno Melo de Souza, Mayara Costa Regazio, Rodolfo Coelho Dalapicola, Claudio Filipi Gon\c{c}alves dos Santos
• Abstract summary: The entire Image Signal Processor (ISP) of a camera relies on several processes to transform the data from the Color Filter Array (CFA) sensor. Deep Learning has emerged as one solution for some of them or even to replace the entire ISP using a single neural network for the task.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The entire Image Signal Processor (ISP) of a camera relies on several processes to transform the data from the Color Filter Array (CFA) sensor, such as demosaicing, denoising, and enhancement. These processes can be executed either by some hardware or via software. In recent years, Deep Learning has emerged as one solution for some of them or even to replace the entire ISP using a single neural network for the task. In this work, we investigated several recent pieces of research in this area and provide deeper analysis and comparison among them, including results and possible points of improvement for future researchers.

Related papers

• Learned Lightweight Smartphone ISP with Unpaired Data [55.2480439325792]
  We propose a novel training method for a learnable Image Signal Processor (ISP)<n>Our unpaired approach employs a multi-term loss function guided by adversarial training.<n>Compared to paired training methods, our unpaired learning strategy shows strong potential and achieves high fidelity.
  arXiv  Detail & Related papers  (2025-05-15T15:37:51Z)
• Deep Learning, Machine Learning -- Digital Signal and Image Processing: From Theory to Application [17.284340134417285]
  Digital Signal Processing (DSP) and Digital Image Processing (DIP) with Machine Learning (ML) and Deep Learning (DL) are popular research areas in Computer Vision. We highlight transformative applications in image enhancement, filtering techniques, and pattern recognition. We implement algorithms that optimize real-time data processing, forming a foundation for scalable, high-performance solutions in computer vision.
  arXiv  Detail & Related papers  (2024-10-27T02:00:33Z)
• DRL-ISP: Multi-Objective Camera ISP with Deep Reinforcement Learning [82.4114562598703]
  We propose a camera ISP framework that utilizes Deep Reinforcement Learning (DRL) and camera ISP toolbox. The proposed DRL-based camera ISP framework iteratively selects a proper tool from the toolbox and applies it to the image to maximize a given vision task-specific reward function. Our proposed DRL-based ISP framework effectively improves the image quality according to each vision task such as RAW-to-RGB image restoration, 2D object detection, and monocular depth estimation.
  arXiv  Detail & Related papers  (2022-07-07T04:34:05Z)
• An Empirical Study of Remote Sensing Pretraining [117.90699699469639]
  We conduct an empirical study of remote sensing pretraining (RSP) on aerial images. RSP can help deliver distinctive performances in scene recognition tasks. RSP mitigates the data discrepancies of traditional ImageNet pretraining on RS images, but it may still suffer from task discrepancies.
  arXiv  Detail & Related papers  (2022-04-06T13:38:11Z)
• ReconfigISP: Reconfigurable Camera Image Processing Pipeline [75.46902933531247]
  Image Signal Processor (ISP) is crucial component in digital cameras that transforms sensor signals into images for us to perceive and understand. Existing ISP designs always adopt a fixed architecture, e.g., several sequential modules connected in a rigid order. In this study, we propose a novel Reconfigurable ISP (ReconfigISP) whose architecture and parameters can be automatically tailored to specific data and tasks.
  arXiv  Detail & Related papers  (2021-09-10T09:56:43Z)
• Beyond Joint Demosaicking and Denoising: An Image Processing Pipeline for a Pixel-bin Image Sensor [0.883717274344425]
  Pixel binning is considered one of the most prominent solutions to tackle the hardware limitation of smartphone cameras. In this paper, we tackle the challenges of joint demosaicing and denoising (JDD) on such an image sensor by introducing a novel learning-based method. The proposed method is guided by a multi-term objective function, including two novel perceptual losses to produce visually plausible images.
  arXiv  Detail & Related papers  (2021-04-19T15:41:28Z)
• A SAR speckle filter based on Residual Convolutional Neural Networks [68.8204255655161]
  This work aims to present a novel method for filtering the speckle noise from Sentinel-1 data by applying Deep Learning (DL) algorithms, based on Convolutional Neural Networks (CNNs) The obtained results, if compared with the state of the art, show a clear improvement in terms of Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM)
  arXiv  Detail & Related papers  (2021-04-19T14:43:07Z)
• PyNET-CA: Enhanced PyNET with Channel Attention for End-to-End Mobile Image Signal Processing [32.7355302269855]
  We propose PyNET-CA, an end-to-end mobile ISP deep learning algorithm for RAW to RGB reconstruction. We demonstrate the performance of the proposed method with comparative experiments and results from the AIM 2020 learned smartphone ISP challenge.
  arXiv  Detail & Related papers  (2021-04-07T03:40:11Z)
• NAS-Navigator: Visual Steering for Explainable One-Shot Deep Neural Network Synthesis [53.106414896248246]
  We present a framework that allows analysts to effectively build the solution sub-graph space and guide the network search by injecting their domain knowledge. Applying this technique in an iterative manner allows analysts to converge to the best performing neural network architecture for a given application.
  arXiv  Detail & Related papers  (2020-09-28T01:48:45Z)
• Fast ultrasonic imaging using end-to-end deep learning [1.0266286487433585]
  Deep neural networks (DNNs) are being used for the data pre-processing and the image post-processing steps separately. We propose a novel deep learning architecture that integrates all three steps to enable end-to-end training.
  arXiv  Detail & Related papers  (2020-09-04T13:53:46Z)
• Accurate RGB-D Salient Object Detection via Collaborative Learning [101.82654054191443]
  RGB-D saliency detection shows impressive ability on some challenge scenarios. We propose a novel collaborative learning framework where edge, depth and saliency are leveraged in a more efficient way.
  arXiv  Detail & Related papers  (2020-07-23T04:33:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.