Fooling Polarization-based Vision using Locally Controllable Polarizing Projection

• URL: http://arxiv.org/abs/2303.17890v2
• Date: Wed, 19 Jun 2024 16:36:22 GMT
• Title: Fooling Polarization-based Vision using Locally Controllable Polarizing Projection
• Authors: Zhuoxiao Li, Zhihang Zhong, Shohei Nobuhara, Ko Nishino, Yinqiang Zheng,
• Abstract summary: We warn the community of the vulnerability of polarization-based vision, which can be more serious than RGB-based vision. By adapting a commercial LCD projector, we achieve locally controllable polarizing projection, which is successfully utilized to fool state-of-the-art polarization-based vision algorithms.
• Score: 55.40484331029597
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Polarization is a fundamental property of light that encodes abundant information regarding surface shape, material, illumination and viewing geometry. The computer vision community has witnessed a blossom of polarization-based vision applications, such as reflection removal, shape-from-polarization, transparent object segmentation and color constancy, partially due to the emergence of single-chip mono/color polarization sensors that make polarization data acquisition easier than ever. However, is polarization-based vision vulnerable to adversarial attacks? If so, is that possible to realize these adversarial attacks in the physical world, without being perceived by human eyes? In this paper, we warn the community of the vulnerability of polarization-based vision, which can be more serious than RGB-based vision. By adapting a commercial LCD projector, we achieve locally controllable polarizing projection, which is successfully utilized to fool state-of-the-art polarization-based vision algorithms for glass segmentation and color constancy. Compared with existing physical attacks on RGB-based vision, which always suffer from the trade-off between attack efficacy and eye conceivability, the adversarial attackers based on polarizing projection are contact-free and visually imperceptible, since naked human eyes can rarely perceive the difference of viciously manipulated polarizing light and ordinary illumination. This poses unprecedented risks on polarization-based vision, both in the monochromatic and trichromatic domain, for which due attentions should be paid and counter measures be considered.

Related papers

• SPIDeRS: Structured Polarization for Invisible Depth and Reflectance Sensing [31.605927493154656]
  We introduce structured polarization for invisible depth and reflectance sensing (SPIDeRS) The key idea is to modulate the angle of linear polarization (AoLP) of projected light at each pixel. The use of polarization makes it invisible and lets us recover not only depth but also directly surface normals and even reflectance.
  arXiv  Detail & Related papers  (2023-12-07T18:59:21Z)
• Neural Polarizer: A Lightweight and Effective Backdoor Defense via Purifying Poisoned Features [62.82817831278743]
  Recent studies have demonstrated the susceptibility of deep neural networks to backdoor attacks. We propose a novel backdoor defense method by inserting a learnable neural polarizer into the backdoored model as an intermediate layer.
  arXiv  Detail & Related papers  (2023-06-29T05:39:58Z)
• PolarFormer: Multi-camera 3D Object Detection with Polar Transformers [93.49713023975727]
  3D object detection in autonomous driving aims to reason "what" and "where" the objects of interest present in a 3D world. Existing methods often adopt the canonical Cartesian coordinate system with perpendicular axis. We propose a new Polar Transformer (PolarFormer) for more accurate 3D object detection in the bird's-eye-view (BEV) taking as input only multi-camera 2D images.
  arXiv  Detail & Related papers  (2022-06-30T16:32:48Z)
• PANDORA: Polarization-Aided Neural Decomposition Of Radiance [20.760987175553655]
  Inverse rendering is a fundamental problem in computer graphics and vision. Recent progress in representing scene properties as coordinate-based neural networks have facilitated neural inverse rendering. We propose PANDORA, a polarimetric inverse rendering approach based on implicit neural representations.
  arXiv  Detail & Related papers  (2022-03-25T05:41:52Z)
• Human Pose and Shape Estimation from Single Polarization Images [45.24275141578927]
  We attempt to estimate human pose and shape from single polarization images by leveraging the polarization-induced geometric cues. A dedicated dataset (PHSPD) is constructed, consisting of over 500K frames with accurate pose and shape annotations. It suggests polarization camera as a promising alternative to the more conventional RGB camera for human pose and shape estimation.
  arXiv  Detail & Related papers  (2021-08-15T22:56:18Z)
• Polarization Guided Specular Reflection Separation [8.280173807482901]
  We derive a polarization guided model to incorporate the polarization information into a designed optimization separation strategy. We show that our method can faithfully separate the specular reflection, especially in some challenging scenarios.
  arXiv  Detail & Related papers  (2021-03-22T08:22:28Z)
• SPAA: Stealthy Projector-based Adversarial Attacks on Deep Image Classifiers [82.19722134082645]
  A stealthy projector-based adversarial attack is proposed in this paper. We approximate the real project-and-capture operation using a deep neural network named PCNet. Our experiments show that the proposed SPAA clearly outperforms other methods by achieving higher attack success rates.
  arXiv  Detail & Related papers  (2020-12-10T18:14:03Z)
• Polarization Human Shape and Pose Dataset [35.156049015251035]
  Polarization images are known to capture polarized reflected lights that preserve rich geometric cues of an object. Inspired by the recent breakthroughs in human shape estimation from a single color image, we attempt to investigate the new question of whether the geometric cues from polarization camera could be leveraged in estimating detailed human body shapes.
  arXiv  Detail & Related papers  (2020-04-30T15:58:21Z)
• Face Anti-Spoofing by Learning Polarization Cues in a Real-World Scenario [50.36920272392624]
  Face anti-spoofing is the key to preventing security breaches in biometric recognition applications. Deep learning method using RGB and infrared images demands a large amount of training data for new attacks. We present a face anti-spoofing method in a real-world scenario by automatic learning the physical characteristics in polarization images of a real face.
  arXiv  Detail & Related papers  (2020-03-18T03:04:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.