Towards RAW Object Detection in Diverse Conditions

• URL: http://arxiv.org/abs/2411.15678v1
• Date: Sun, 24 Nov 2024 01:23:04 GMT
• Title: Towards RAW Object Detection in Diverse Conditions
• Authors: Zhong-Yu Li, Xin Jin, Boyuan Sun, Chun-Le Guo, Ming-Ming Cheng,
• Abstract summary: We introduce the AODRaw dataset, which offers 7,785 high-resolution real RAW images with 135,601 annotated instances spanning 62 categories. We find that sRGB pre-training constrains the potential of RAW object detection due to the domain gap between sRGB and RAW. We distill the knowledge from an off-the-shelf model pre-trained on the sRGB domain to assist RAW pre-training.
• Score: 65.30190654593842
• License:
• Abstract: Existing object detection methods often consider sRGB input, which was compressed from RAW data using ISP originally designed for visualization. However, such compression might lose crucial information for detection, especially under complex light and weather conditions. We introduce the AODRaw dataset, which offers 7,785 high-resolution real RAW images with 135,601 annotated instances spanning 62 categories, capturing a broad range of indoor and outdoor scenes under 9 distinct light and weather conditions. Based on AODRaw that supports RAW and sRGB object detection, we provide a comprehensive benchmark for evaluating current detection methods. We find that sRGB pre-training constrains the potential of RAW object detection due to the domain gap between sRGB and RAW, prompting us to directly pre-train on the RAW domain. However, it is harder for RAW pre-training to learn rich representations than sRGB pre-training due to the camera noise. To assist RAW pre-training, we distill the knowledge from an off-the-shelf model pre-trained on the sRGB domain. As a result, we achieve substantial improvements under diverse and adverse conditions without relying on extra pre-processing modules. Code and dataset are available at https://github.com/lzyhha/AODRaw.

Related papers

• RAW-Diffusion: RGB-Guided Diffusion Models for High-Fidelity RAW Image Generation [4.625376287612609]
  We propose a novel diffusion-based method for generating RAW images guided by RGB images. This approach yields high-fidelity RAW images, enabling the creation of camera-specific RAW datasets. We extend our method to create BDD100K-RAW and Cityscapes-RAW datasets, revealing its effectiveness for object detection in RAW imagery.
  arXiv  Detail & Related papers  (2024-11-20T09:40:12Z)
• Unveiling Hidden Details: A RAW Data-Enhanced Paradigm for Real-World Super-Resolution [56.98910228239627]
  Real-world image super-resolution (Real SR) aims to generate high-fidelity, detail-rich high-resolution (HR) images from low-resolution (LR) counterparts. Existing Real SR methods primarily focus on generating details from the LR RGB domain, often leading to a lack of richness or fidelity in fine details. We pioneer the use of details hidden in RAW data to complement existing RGB-only methods, yielding superior outputs.
  arXiv  Detail & Related papers  (2024-11-16T13:29:50Z)
• Toward Efficient Deep Blind RAW Image Restoration [56.41827271721955]
  We design a new realistic degradation pipeline for training deep blind RAW restoration models. Our pipeline considers realistic sensor noise, motion blur, camera shake, and other common degradations. The models trained with our pipeline and data from multiple sensors, can successfully reduce noise and blur, and recover details in RAW images captured from different cameras.
  arXiv  Detail & Related papers  (2024-09-26T18:34:37Z)
• Raw Instinct: Trust Your Classifiers and Skip the Conversion [12.323236593352698]
  We show that a sufficiently advanced classifier can yield equivalent results on RAW input compared to RGB. We furthermore show that the total computation time from RAW image data to classification results for RAW images can be up to 8.46 times faster than RGB.
  arXiv  Detail & Related papers  (2024-03-21T14:45:41Z)
• BSRAW: Improving Blind RAW Image Super-Resolution [63.408484584265985]
  We tackle blind image super-resolution in the RAW domain. We design a realistic degradation pipeline tailored specifically for training models with raw sensor data. Our BSRAW models trained with our pipeline can upscale real-scene RAW images and improve their quality.
  arXiv  Detail & Related papers  (2023-12-24T14:17:28Z)
• Self-Supervised Reversed Image Signal Processing via Reference-Guided Dynamic Parameter Selection [1.1602089225841632]
  We propose a self-supervised reversed ISP method that does not require metadata and paired images. The proposed method converts a RGB image into a RAW-like image taken in the same environment with the same sensor as a reference RAW image. We show that the proposed method is able to learn various reversed ISPs with comparable accuracy to other state-of-the-art supervised methods.
  arXiv  Detail & Related papers  (2023-03-24T11:12:05Z)
• Efficient Visual Computing with Camera RAW Snapshots [41.9863557302409]
  Conventional cameras capture image irradiance on a sensor and convert it to RGB images using an image signal processor (ISP) One can argue that since RAW images contain all the captured information, the conversion of RAW to RGB using an ISP is not necessary for visual computing. We propose a novel $rho$-Vision framework to perform high-level semantic understanding and low-level compression using RAW images.
  arXiv  Detail & Related papers  (2022-12-15T12:54:21Z)
• Towards Low Light Enhancement with RAW Images [101.35754364753409]
  We make the first benchmark effort to elaborate on the superiority of using RAW images in the low light enhancement. We develop a new evaluation framework, Factorized Enhancement Model (FEM), which decomposes the properties of RAW images into measurable factors. A RAW-guiding Exposure Enhancement Network (REENet) is developed, which makes trade-offs between the advantages and inaccessibility of RAW images in real applications.
  arXiv  Detail & Related papers  (2021-12-28T07:27:51Z)
• Learning RAW-to-sRGB Mappings with Inaccurately Aligned Supervision [76.41657124981549]
  This paper presents a joint learning model for image alignment and RAW-to-sRGB mapping. Experiments show that our method performs favorably against state-of-the-arts on ZRR and SR-RAW datasets.
  arXiv  Detail & Related papers  (2021-08-18T12:41:36Z)
• Invertible Image Signal Processing [42.109752151834456]
  Invertible Image Signal Processing (InvISP) pipeline enables rendering visually appealing sRGB images. We can reconstruct realistic RAW data instead of synthesizing RAW data from sRGB images without any memory overhead.
  arXiv  Detail & Related papers  (2021-03-28T06:30:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.