PQDynamicISP: Dynamically Controlled Image Signal Processor for Any Image Sensors Pursuing Perceptual Quality

• URL: http://arxiv.org/abs/2403.10091v1
• Date: Fri, 15 Mar 2024 08:08:24 GMT
• Title: PQDynamicISP: Dynamically Controlled Image Signal Processor for Any Image Sensors Pursuing Perceptual Quality
• Authors: Masakazu Yoshimura, Junji Otsuka, Takeshi Ohashi,
• Abstract summary: Instead of tuning the parameters of the ISP, we propose to control them dynamically for each environment and even locally. Our method can process different image sensors with a single ISP through dynamic control, whereas conventional methods require training for each sensor.
• Score: 0.5530212768657544
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Full DNN-based image signal processors (ISPs) have been actively studied and have achieved superior image quality compared to conventional ISPs. In contrast to this trend, we propose a lightweight ISP that consists of simple conventional ISP functions but achieves high image quality by increasing expressiveness. Specifically, instead of tuning the parameters of the ISP, we propose to control them dynamically for each environment and even locally. As a result, state-of-the-art accuracy is achieved on various datasets, including other tasks like tone mapping and image enhancement, even though ours is lighter than DNN-based ISPs. Additionally, our method can process different image sensors with a single ISP through dynamic control, whereas conventional methods require training for each sensor.

Related papers

• Towards Realistic Low-Light Image Enhancement via ISP Driven Data Modeling [61.95831392879045]
  Deep neural networks (DNNs) have recently become the leading method for low-light image enhancement (LLIE) Despite significant progress, their outputs may still exhibit issues such as amplified noise, incorrect white balance, or unnatural enhancements when deployed in real world applications. A key challenge is the lack of diverse, large scale training data that captures the complexities of low-light conditions and imaging pipelines. We propose a novel image signal processing (ISP) driven data synthesis pipeline that addresses these challenges by generating unlimited paired training data.
  arXiv  Detail & Related papers  (2025-04-16T15:53:53Z)
• AdaptiveISP: Learning an Adaptive Image Signal Processor for Object Detection [15.63212587981912]
  Image Signal Processors (ISPs) convert raw sensor signals into digital images. ISPs are designed to maximize the image quality, which are sub-optimal in the performance of high-level computer vision tasks. We propose AdaptiveISP, a task-driven and scene-adaptive ISP.
  arXiv  Detail & Related papers  (2024-10-30T11:49:06Z)
• Simple Image Signal Processing using Global Context Guidance [56.41827271721955]
  Deep learning-based ISPs aim to transform RAW images into DSLR-like RGB images using deep neural networks. We propose a novel module that can be integrated into any neural ISP to capture the global context information from the full RAW images. Our model achieves state-of-the-art results on different benchmarks using diverse and real smartphone images.
  arXiv  Detail & Related papers  (2024-04-17T17:11:47Z)
• Learning Degradation-Independent Representations for Camera ISP Pipelines [14.195578257521934]
  We propose a novel approach to learn degradation-independent representations (DiR) through the refinement of a self-supervised learned baseline representation. The proposed DiR learning technique has remarkable domain generalization capability and it outperforms state-of-the-art methods across various downstream tasks.
  arXiv  Detail & Related papers  (2023-07-03T05:38:28Z)
• DynamicISP: Dynamically Controlled Image Signal Processor for Image Recognition [0.5530212768657544]
  "DynamicISP," consists of multiple classical ISP functions and dynamically controls the parameters of each frame according to recognition result of previous frame. We show our method successfully controls the parameters of multiple ISP functions and achieves state-of-the-art accuracy with low computational cost in single and multi-category object detection tasks.
  arXiv  Detail & Related papers  (2022-11-02T14:22:50Z)
• Image sensing with multilayer, nonlinear optical neural networks [4.252754174399026]
  An emerging image-sensing paradigm breaks this delineation between data collection and analysis. By optically encoding images into a compressed, low-dimensional latent space suitable for efficient post-analysis, these image sensors can operate with fewer pixels and fewer photons. We demonstrate a multilayer ONN pre-processor for image sensing, using a commercial image intensifier as a parallel optoelectronic, optical-to-optical nonlinear activation function.
  arXiv  Detail & Related papers  (2022-07-27T21:00:31Z)
• Controllable Image Enhancement [66.18525728881711]
  We present a semiautomatic image enhancement algorithm that can generate high-quality images with multiple styles by controlling a few parameters. An encoder-decoder framework encodes the retouching skills into latent codes and decodes them into the parameters of image signal processing functions.
  arXiv  Detail & Related papers  (2022-06-16T23:54:53Z)
• Multi-Content Complementation Network for Salient Object Detection in Optical Remote Sensing Images [108.79667788962425]
  salient object detection in optical remote sensing images (RSI-SOD) remains to be a challenging emerging topic. We propose a novel Multi-Content Complementation Network (MCCNet) to explore the complementarity of multiple content for RSI-SOD. In MCCM, we consider multiple types of features that are critical to RSI-SOD, including foreground features, edge features, background features, and global image-level features.
  arXiv  Detail & Related papers  (2021-12-02T04:46:40Z)
• Dynamic Slimmable Denoising Network [64.77565006158895]
  Dynamic slimmable denoising network (DDSNet) is a general method to achieve good denoising quality with less computational complexity. OurNet is empowered with the ability of dynamic inference by a dynamic gate. Our experiments demonstrate our-Net consistently outperforms the state-of-the-art individually trained static denoising networks.
  arXiv  Detail & Related papers  (2021-10-17T22:45:33Z)
• ReconfigISP: Reconfigurable Camera Image Processing Pipeline [75.46902933531247]
  Image Signal Processor (ISP) is crucial component in digital cameras that transforms sensor signals into images for us to perceive and understand. Existing ISP designs always adopt a fixed architecture, e.g., several sequential modules connected in a rigid order. In this study, we propose a novel Reconfigurable ISP (ReconfigISP) whose architecture and parameters can be automatically tailored to specific data and tasks.
  arXiv  Detail & Related papers  (2021-09-10T09:56:43Z)
• Pseudo-ISP: Learning Pseudo In-camera Signal Processing Pipeline from A Color Image Denoiser [73.88164217509816]
  We present an unpaired learning scheme to adapt a color image denoiser for handling test images with noise discrepancy. We consider a practical training setting, i.e., a pre-trained denoiser, a set of test noisy images, and an unpaired set of clean images. Pseudo-ISP is effective in synthesizing realistic noisy sRGB images.
  arXiv  Detail & Related papers  (2021-03-18T13:11:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.