DRL-ISP: Multi-Objective Camera ISP with Deep Reinforcement Learning
- URL: http://arxiv.org/abs/2207.03081v1
- Date: Thu, 7 Jul 2022 04:34:05 GMT
- Title: DRL-ISP: Multi-Objective Camera ISP with Deep Reinforcement Learning
- Authors: Ukcheol Shin, Kyunghyun Lee, In So Kweon
- Abstract summary: We propose a camera ISP framework that utilizes Deep Reinforcement Learning (DRL) and camera ISP toolbox.
The proposed DRL-based camera ISP framework iteratively selects a proper tool from the toolbox and applies it to the image to maximize a given vision task-specific reward function.
Our proposed DRL-based ISP framework effectively improves the image quality according to each vision task such as RAW-to-RGB image restoration, 2D object detection, and monocular depth estimation.
- Score: 82.4114562598703
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: In this paper, we propose a multi-objective camera ISP framework that
utilizes Deep Reinforcement Learning (DRL) and camera ISP toolbox that consist
of network-based and conventional ISP tools. The proposed DRL-based camera ISP
framework iteratively selects a proper tool from the toolbox and applies it to
the image to maximize a given vision task-specific reward function. For this
purpose, we implement total 51 ISP tools that include exposure correction,
color-and-tone correction, white balance, sharpening, denoising, and the
others. We also propose an efficient DRL network architecture that can extract
the various aspects of an image and make a rigid mapping relationship between
images and a large number of actions. Our proposed DRL-based ISP framework
effectively improves the image quality according to each vision task such as
RAW-to-RGB image restoration, 2D object detection, and monocular depth
estimation.
Related papers
- Parameter-Inverted Image Pyramid Networks [49.35689698870247]
We propose a novel network architecture known as the Inverted Image Pyramid Networks (PIIP)
Our core idea is to use models with different parameter sizes to process different resolution levels of the image pyramid.
PIIP achieves superior performance in tasks such as object detection, segmentation, and image classification.
arXiv Detail & Related papers (2024-06-06T17:59:10Z) - LW-ISP: A Lightweight Model with ISP and Deep Learning [17.972611191715888]
We show the possibility of learning-based method to achieve real-time high-performance processing in the ISP pipeline.
We propose LW-ISP, a novel architecture designed to implicitly learn the image mapping from RAW data to RGB image.
Experiments demonstrate that LW-ISP has achieved a 0.38 dB improvement in PSNR compared to the previous best method.
arXiv Detail & Related papers (2022-10-08T04:00:03Z) - Learning Deep Context-Sensitive Decomposition for Low-Light Image
Enhancement [58.72667941107544]
A typical framework is to simultaneously estimate the illumination and reflectance, but they disregard the scene-level contextual information encapsulated in feature spaces.
We develop a new context-sensitive decomposition network architecture to exploit the scene-level contextual dependencies on spatial scales.
We develop a lightweight CSDNet (named LiteCSDNet) by reducing the number of channels.
arXiv Detail & Related papers (2021-12-09T06:25:30Z) - ReconfigISP: Reconfigurable Camera Image Processing Pipeline [75.46902933531247]
Image Signal Processor (ISP) is crucial component in digital cameras that transforms sensor signals into images for us to perceive and understand.
Existing ISP designs always adopt a fixed architecture, e.g., several sequential modules connected in a rigid order.
In this study, we propose a novel Reconfigurable ISP (ReconfigISP) whose architecture and parameters can be automatically tailored to specific data and tasks.
arXiv Detail & Related papers (2021-09-10T09:56:43Z) - Robust super-resolution depth imaging via a multi-feature fusion deep
network [2.351601888896043]
Light detection and ranging (LIDAR) via single-photon sensitive detector (SPAD) arrays is an emerging technology that enables the acquisition of depth images at high frame rates.
We develop a deep network built specifically to take advantage of the multiple features that can be extracted from a camera's histogram data.
We apply the network to a range of 3D data, demonstrating denoising and a four-fold resolution enhancement of depth.
arXiv Detail & Related papers (2020-11-20T14:24:12Z) - PlenoptiCam v1.0: A light-field imaging framework [8.467466998915018]
Light-field cameras play a vital role for rich 3-D information retrieval in narrow range depth sensing applications.
Key obstacle in composing light-fields from exposures taken by a plenoptic camera is to calibrate computationally, align and rearrange four-dimensional image data.
Several attempts have been proposed to enhance the overall image quality by tailoring pipelines dedicated to particular plenoptic cameras.
arXiv Detail & Related papers (2020-10-14T09:23:18Z) - AWNet: Attentive Wavelet Network for Image ISP [14.58067200317891]
We introduce a novel network that utilizes the attention mechanism and wavelet transform, dubbed AWNet, to tackle this learnable image ISP problem.
Our proposed method enables us to restore favorable image details from RAW information and achieve a larger receptive field.
Experimental results indicate the advances of our design in both qualitative and quantitative measurements.
arXiv Detail & Related papers (2020-08-20T23:28:41Z) - Deep 3D Capture: Geometry and Reflectance from Sparse Multi-View Images [59.906948203578544]
We introduce a novel learning-based method to reconstruct the high-quality geometry and complex, spatially-varying BRDF of an arbitrary object.
We first estimate per-view depth maps using a deep multi-view stereo network.
These depth maps are used to coarsely align the different views.
We propose a novel multi-view reflectance estimation network architecture.
arXiv Detail & Related papers (2020-03-27T21:28:54Z) - Learning Enriched Features for Real Image Restoration and Enhancement [166.17296369600774]
convolutional neural networks (CNNs) have achieved dramatic improvements over conventional approaches for image restoration task.
We present a novel architecture with the collective goals of maintaining spatially-precise high-resolution representations through the entire network.
Our approach learns an enriched set of features that combines contextual information from multiple scales, while simultaneously preserving the high-resolution spatial details.
arXiv Detail & Related papers (2020-03-15T11:04:30Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.