All You Need is RAW: Defending Against Adversarial Attacks with Camera Image Pipelines

• URL: http://arxiv.org/abs/2112.09219v1
• Date: Thu, 16 Dec 2021 21:54:26 GMT
• Title: All You Need is RAW: Defending Against Adversarial Attacks with Camera Image Pipelines
• Authors: Yuxuan Zhang, Bo Dong, Felix Heide
• Abstract summary: We propose a model-agnostic adversarial defensive method for image-to-image mapping. The method maps the input RGB images to Bayer RAW space and back to output RGB using a learned camera image signal processing pipeline. As a result, the method generalizes to unseen tasks without additional retraining.
• Score: 31.043289921613933
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Existing neural networks for computer vision tasks are vulnerable to adversarial attacks: adding imperceptible perturbations to the input images can fool these methods to make a false prediction on an image that was correctly predicted without the perturbation. Various defense methods have proposed image-to-image mapping methods, either including these perturbations in the training process or removing them in a preprocessing denoising step. In doing so, existing methods often ignore that the natural RGB images in today's datasets are not captured but, in fact, recovered from RAW color filter array captures that are subject to various degradations in the capture. In this work, we exploit this RAW data distribution as an empirical prior for adversarial defense. Specifically, we proposed a model-agnostic adversarial defensive method, which maps the input RGB images to Bayer RAW space and back to output RGB using a learned camera image signal processing (ISP) pipeline to eliminate potential adversarial patterns. The proposed method acts as an off-the-shelf preprocessing module and, unlike model-specific adversarial training methods, does not require adversarial images to train. As a result, the method generalizes to unseen tasks without additional retraining. Experiments on large-scale datasets (e.g., ImageNet, COCO) for different vision tasks (e.g., classification, semantic segmentation, object detection) validate that the method significantly outperforms existing methods across task domains.

Related papers

• Learning to Rank Patches for Unbiased Image Redundancy Reduction [80.93989115541966]
  Images suffer from heavy spatial redundancy because pixels in neighboring regions are spatially correlated. Existing approaches strive to overcome this limitation by reducing less meaningful image regions. We propose a self-supervised framework for image redundancy reduction called Learning to Rank Patches.
  arXiv  Detail & Related papers  (2024-03-31T13:12:41Z)
• Joint Demosaicing and Denoising with Double Deep Image Priors [5.3686304202729]
  Demosaicing and denoising RAW images are crucial steps in the processing pipeline of modern digital cameras. Recent deep neural-network-based approaches have shown the effectiveness of joint demosaicing and denoising to mitigate such challenges. We propose a novel joint demosaicing and denoising method, dubbed JDD-DoubleDIP, which operates directly on a single RAW image without requiring any training data.
  arXiv  Detail & Related papers  (2023-09-18T01:53:10Z)
• Self-Supervised Reversed Image Signal Processing via Reference-Guided Dynamic Parameter Selection [1.1602089225841632]
  We propose a self-supervised reversed ISP method that does not require metadata and paired images. The proposed method converts a RGB image into a RAW-like image taken in the same environment with the same sensor as a reference RAW image. We show that the proposed method is able to learn various reversed ISPs with comparable accuracy to other state-of-the-art supervised methods.
  arXiv  Detail & Related papers  (2023-03-24T11:12:05Z)
• Adversarial RAW: Image-Scaling Attack Against Imaging Pipeline [5.036532914308395]
  In this paper, we develop an image-scaling attack targeting on ISP pipeline, where the crafted adversarial RAW can be transformed into attack image. To make the adversarial attack more applicable, we consider the gradient-unavailable ISP pipeline, in which a proxy model that well learns the RAW-to-RGB transformations is proposed as the gradient oracles.
  arXiv  Detail & Related papers  (2022-06-02T07:35:50Z)
• Diffusion Models for Adversarial Purification [69.1882221038846]
  Adrial purification refers to a class of defense methods that remove adversarial perturbations using a generative model. We propose DiffPure that uses diffusion models for adversarial purification. Our method achieves the state-of-the-art results, outperforming current adversarial training and adversarial purification methods.
  arXiv  Detail & Related papers  (2022-05-16T06:03:00Z)
• Transform your Smartphone into a DSLR Camera: Learning the ISP in the Wild [159.71025525493354]
  We propose a trainable Image Signal Processing framework that produces DSLR quality images given RAW images captured by a smartphone. To address the color misalignments between training image pairs, we employ a color-conditional ISP network and optimize a novel parametric color mapping between each input RAW and reference DSLR image.
  arXiv  Detail & Related papers  (2022-03-20T20:13:59Z)
• Model-Based Image Signal Processors via Learnable Dictionaries [6.766416093990318]
  Digital cameras transform sensor RAW readings into RGB images by means of their Image Signal Processor (ISP) Recent approaches have attempted to bridge this gap by estimating the RGB to RAW mapping. We present a novel hybrid model-based and data-driven ISP that is both learnable and interpretable.
  arXiv  Detail & Related papers  (2022-01-10T08:36:10Z)
• Inverse Problems Leveraging Pre-trained Contrastive Representations [88.70821497369785]
  We study a new family of inverse problems for recovering representations of corrupted data. We propose a supervised inversion method that uses a contrastive objective to obtain excellent representations for highly corrupted images. Our method outperforms end-to-end baselines even with a fraction of the labeled data in a wide range of forward operators.
  arXiv  Detail & Related papers  (2021-10-14T15:06:30Z)
• Learning RAW-to-sRGB Mappings with Inaccurately Aligned Supervision [76.41657124981549]
  This paper presents a joint learning model for image alignment and RAW-to-sRGB mapping. Experiments show that our method performs favorably against state-of-the-arts on ZRR and SR-RAW datasets.
  arXiv  Detail & Related papers  (2021-08-18T12:41:36Z)
• Learning to Jointly Deblur, Demosaick and Denoise Raw Images [39.149955856017876]
  We adapt an existing learning-based approach to RGB image deblurring to handle raw images by introducing a new interpretable module. We train this model on RGB images converted into raw ones following a realistic invertible camera pipeline. We also apply our approach to remove a camera's inherent blur from real images, in essence deblurring sharp images.
  arXiv  Detail & Related papers  (2021-04-13T19:02:59Z)
• CycleISP: Real Image Restoration via Improved Data Synthesis [166.17296369600774]
  We present a framework that models camera imaging pipeline in forward and reverse directions. By training a new image denoising network on realistic synthetic data, we achieve the state-of-the-art performance on real camera benchmark datasets.
  arXiv  Detail & Related papers  (2020-03-17T15:20:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.