Beyond RGB: Adaptive Parallel Processing for RAW Object Detection

• URL: http://arxiv.org/abs/2503.13163v1
• Date: Mon, 17 Mar 2025 13:36:49 GMT
• Title: Beyond RGB: Adaptive Parallel Processing for RAW Object Detection
• Authors: Shani Gamrian, Hila Barel, Feiran Li, Masakazu Yoshimura, Daisuke Iso,
• Abstract summary: Raw Adaptation Module (RAM) is a module designed to replace the traditional Image Signal Processing (ISP)<n>Our approach outperforms RGB-based methods and achieves state-of-the-art results across diverse RAW image datasets.
• Score: 5.36869872375791
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Object detection models are typically applied to standard RGB images processed through Image Signal Processing (ISP) pipelines, which are designed to enhance sensor-captured RAW images for human vision. However, these ISP functions can lead to a loss of critical information that may be essential in optimizing for computer vision tasks, such as object detection. In this work, we introduce Raw Adaptation Module (RAM), a module designed to replace the traditional ISP, with parameters optimized specifically for RAW object detection. Inspired by the parallel processing mechanisms of the human visual system, RAM departs from existing learned ISP methods by applying multiple ISP functions in parallel rather than sequentially, allowing for a more comprehensive capture of image features. These processed representations are then fused in a specialized module, which dynamically integrates and optimizes the information for the target task. This novel approach not only leverages the full potential of RAW sensor data but also enables task-specific pre-processing, resulting in superior object detection performance. Our approach outperforms RGB-based methods and achieves state-of-the-art results across diverse RAW image datasets under varying lighting conditions and dynamic ranges.

Related papers

• RAW-Adapter: Adapting Pre-trained Visual Model to Camera RAW Images and A Benchmark [58.47710845549269]
  RAW-Adapter is a novel framework that incorporates learnable ISP modules as input-level adapters to adjust RAW inputs. RAW-Adapter serves as a general framework applicable to various computer vision frameworks. We introduce RAW-Bench, which incorporates 17 types of RAW-based common corruptions.
  arXiv  Detail & Related papers  (2025-03-21T10:37:42Z)
• Keypoint Detection and Description for Raw Bayer Images [10.443350617606972]
  Keypoint detection and local feature description are fundamental tasks in robotic perception, critical for applications such as SLAM, robot localization, feature matching, pose estimation, and 3D mapping. While existing methods predominantly operate on RGB images, we propose a novel network that directly processes raw images, bypassing the need for the Image Signal Processor (ISP). This work represents the first attempt to develop a keypoint detection and feature description network specifically for raw images, offering a more efficient solution for resource-constrained environments.
  arXiv  Detail & Related papers  (2025-03-11T17:54:12Z)
• SimROD: A Simple Baseline for Raw Object Detection with Global and Local Enhancements [7.08243476424994]
  We propose SimROD, a lightweight and effective approach for RAW object detection.<n>We introduce a Global Gamma Enhancement (GGE) module, which applies a learnable global gamma transformation with only four parameters.<n>Our work highlights the potential of RAW data for real-world object detection.
  arXiv  Detail & Related papers  (2025-03-10T09:23:14Z)
• RAWMamba: Unified sRGB-to-RAW De-rendering With State Space Model [52.250939617273744]
  We propose RAWMamba, a Mamba-based unified framework for sRGB-to-RAW de-rendering. The core of RAWMamba is the Unified Metadata Embedding (UME) module, which harmonizes diverse metadata types into a unified representation. The Local Tone-Aware Mamba module captures long-range dependencies to enable effective global propagation of metadata.
  arXiv  Detail & Related papers  (2024-11-18T16:45:44Z)
• A Learnable Color Correction Matrix for RAW Reconstruction [19.394856071610604]
  We introduce a learnable color correction matrix (CCM) to approximate the complex inverse image signal processor (ISP) Experimental results demonstrate that simulated RAW (simRAW) images generated by our method provide performance improvements equivalent to those produced by more complex inverse ISP methods.
  arXiv  Detail & Related papers  (2024-09-04T07:46:42Z)
• RAW-Adapter: Adapting Pre-trained Visual Model to Camera RAW Images [51.68432586065828]
  We introduce RAW-Adapter, a novel approach aimed at adapting sRGB pre-trained models to camera RAW data. Raw-Adapter comprises input-level adapters that employ learnable ISP stages to adjust RAW inputs, as well as model-level adapters to build connections between ISP stages and subsequent high-level networks.
  arXiv  Detail & Related papers  (2024-08-27T06:14:54Z)
• Beyond Learned Metadata-based Raw Image Reconstruction [86.1667769209103]
  Raw images have distinct advantages over sRGB images, e.g., linearity and fine-grained quantization levels. They are not widely adopted by general users due to their substantial storage requirements. We propose a novel framework that learns a compact representation in the latent space, serving as metadata.
  arXiv  Detail & Related papers  (2023-06-21T06:59:07Z)
• Self-Supervised Reversed Image Signal Processing via Reference-Guided Dynamic Parameter Selection [1.1602089225841632]
  We propose a self-supervised reversed ISP method that does not require metadata and paired images. The proposed method converts a RGB image into a RAW-like image taken in the same environment with the same sensor as a reference RAW image. We show that the proposed method is able to learn various reversed ISPs with comparable accuracy to other state-of-the-art supervised methods.
  arXiv  Detail & Related papers  (2023-03-24T11:12:05Z)
• Raw Image Reconstruction with Learned Compact Metadata [61.62454853089346]
  We propose a novel framework to learn a compact representation in the latent space serving as the metadata in an end-to-end manner. We show how the proposed raw image compression scheme can adaptively allocate more bits to image regions that are important from a global perspective.
  arXiv  Detail & Related papers  (2023-02-25T05:29:45Z)
• Efficient Visual Computing with Camera RAW Snapshots [41.9863557302409]
  Conventional cameras capture image irradiance on a sensor and convert it to RGB images using an image signal processor (ISP) One can argue that since RAW images contain all the captured information, the conversion of RAW to RGB using an ISP is not necessary for visual computing. We propose a novel $rho$-Vision framework to perform high-level semantic understanding and low-level compression using RAW images.
  arXiv  Detail & Related papers  (2022-12-15T12:54:21Z)
• Reversed Image Signal Processing and RAW Reconstruction. AIM 2022 Challenge Report [109.2135194765743]
  This paper introduces the AIM 2022 Challenge on Reversed Image Signal Processing and RAW Reconstruction. We aim to recover raw sensor images from the corresponding RGBs without metadata and, by doing this, "reverse" the ISP transformation.
  arXiv  Detail & Related papers  (2022-10-20T10:43:53Z)
• Model-Based Image Signal Processors via Learnable Dictionaries [6.766416093990318]
  Digital cameras transform sensor RAW readings into RGB images by means of their Image Signal Processor (ISP) Recent approaches have attempted to bridge this gap by estimating the RGB to RAW mapping. We present a novel hybrid model-based and data-driven ISP that is both learnable and interpretable.
  arXiv  Detail & Related papers  (2022-01-10T08:36:10Z)
• Towards Low Light Enhancement with RAW Images [101.35754364753409]
  We make the first benchmark effort to elaborate on the superiority of using RAW images in the low light enhancement. We develop a new evaluation framework, Factorized Enhancement Model (FEM), which decomposes the properties of RAW images into measurable factors. A RAW-guiding Exposure Enhancement Network (REENet) is developed, which makes trade-offs between the advantages and inaccessibility of RAW images in real applications.
  arXiv  Detail & Related papers  (2021-12-28T07:27:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.