DynamicISP: Dynamically Controlled Image Signal Processor for Image Recognition

• URL: http://arxiv.org/abs/2211.01146v3
• Date: Mon, 28 Aug 2023 02:59:24 GMT
• Title: DynamicISP: Dynamically Controlled Image Signal Processor for Image Recognition
• Authors: Masakazu Yoshimura, Junji Otsuka, Atsushi Irie, Takeshi Ohashi
• Abstract summary: "DynamicISP," consists of multiple classical ISP functions and dynamically controls the parameters of each frame according to recognition result of previous frame. We show our method successfully controls the parameters of multiple ISP functions and achieves state-of-the-art accuracy with low computational cost in single and multi-category object detection tasks.
• Score: 0.5530212768657544
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Image Signal Processors (ISPs) play important roles in image recognition tasks as well as in the perceptual quality of captured images. In most cases, experts make a lot of effort to manually tune many parameters of ISPs, but the parameters are sub-optimal. In the literature, two types of techniques have been actively studied: a machine learning-based parameter tuning technique and a DNN-based ISP technique. The former is lightweight but lacks expressive power. The latter has expressive power, but the computational cost is too heavy on edge devices. To solve these problems, we propose "DynamicISP," which consists of multiple classical ISP functions and dynamically controls the parameters of each frame according to the recognition result of the previous frame. We show our method successfully controls the parameters of multiple ISP functions and achieves state-of-the-art accuracy with low computational cost in single and multi-category object detection tasks.

Related papers

• Striving for Faster and Better: A One-Layer Architecture with Auto Re-parameterization for Low-Light Image Enhancement [50.93686436282772]
  We aim to delve into the limits of image enhancers both from visual quality and computational efficiency. By rethinking the task demands, we build an explicit connection, i.e., visual quality and computational efficiency are corresponding to model learning and structure design. Ultimately, this achieves efficient low-light image enhancement using only a single convolutional layer, while maintaining excellent visual quality.
  arXiv  Detail & Related papers  (2025-02-27T08:20:03Z)
• AdaptiveISP: Learning an Adaptive Image Signal Processor for Object Detection [15.63212587981912]
  Image Signal Processors (ISPs) convert raw sensor signals into digital images. ISPs are designed to maximize the image quality, which are sub-optimal in the performance of high-level computer vision tasks. We propose AdaptiveISP, a task-driven and scene-adaptive ISP.
  arXiv  Detail & Related papers  (2024-10-30T11:49:06Z)
• Parameter-Inverted Image Pyramid Networks [49.35689698870247]
  We propose a novel network architecture known as the Inverted Image Pyramid Networks (PIIP) Our core idea is to use models with different parameter sizes to process different resolution levels of the image pyramid. PIIP achieves superior performance in tasks such as object detection, segmentation, and image classification.
  arXiv  Detail & Related papers  (2024-06-06T17:59:10Z)
• Simple Image Signal Processing using Global Context Guidance [56.41827271721955]
  Deep learning-based ISPs aim to transform RAW images into DSLR-like RGB images using deep neural networks. We propose a novel module that can be integrated into any neural ISP to capture the global context information from the full RAW images. Our model achieves state-of-the-art results on different benchmarks using diverse and real smartphone images.
  arXiv  Detail & Related papers  (2024-04-17T17:11:47Z)
• PQDynamicISP: Dynamically Controlled Image Signal Processor for Any Image Sensors Pursuing Perceptual Quality [0.5530212768657544]
  Instead of tuning the parameters of the ISP, we propose to control them dynamically for each environment and even locally. Our method can process different image sensors with a single ISP through dynamic control, whereas conventional methods require training for each sensor.
  arXiv  Detail & Related papers  (2024-03-15T08:08:24Z)
• Learning to Compose SuperWeights for Neural Parameter Allocation Search [61.078949532440724]
  We show that our approach can generate parameters for many network using the same set of weights. This enables us to support tasks like efficient ensembling and anytime prediction.
  arXiv  Detail & Related papers  (2023-12-03T04:20:02Z)
• DRL-ISP: Multi-Objective Camera ISP with Deep Reinforcement Learning [82.4114562598703]
  We propose a camera ISP framework that utilizes Deep Reinforcement Learning (DRL) and camera ISP toolbox. The proposed DRL-based camera ISP framework iteratively selects a proper tool from the toolbox and applies it to the image to maximize a given vision task-specific reward function. Our proposed DRL-based ISP framework effectively improves the image quality according to each vision task such as RAW-to-RGB image restoration, 2D object detection, and monocular depth estimation.
  arXiv  Detail & Related papers  (2022-07-07T04:34:05Z)
• Controllable Image Enhancement [66.18525728881711]
  We present a semiautomatic image enhancement algorithm that can generate high-quality images with multiple styles by controlling a few parameters. An encoder-decoder framework encodes the retouching skills into latent codes and decodes them into the parameters of image signal processing functions.
  arXiv  Detail & Related papers  (2022-06-16T23:54:53Z)
• ReconfigISP: Reconfigurable Camera Image Processing Pipeline [75.46902933531247]
  Image Signal Processor (ISP) is crucial component in digital cameras that transforms sensor signals into images for us to perceive and understand. Existing ISP designs always adopt a fixed architecture, e.g., several sequential modules connected in a rigid order. In this study, we propose a novel Reconfigurable ISP (ReconfigISP) whose architecture and parameters can be automatically tailored to specific data and tasks.
  arXiv  Detail & Related papers  (2021-09-10T09:56:43Z)
• TFPnP: Tuning-free Plug-and-Play Proximal Algorithm with Applications to Inverse Imaging Problems [22.239477171296056]
  Plug-and-Play (MM) is a non- optimization framework that combines numerical algorithms, for example, with advanced denoising priors. We discuss several practical considerations of more denoisers, which together with our learned strategies are state-of-the-art results.
  arXiv  Detail & Related papers  (2020-11-18T14:19:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.