PBCAT: Patch-based composite adversarial training against physically realizable attacks on object detection

• URL: http://arxiv.org/abs/2506.23581v2
• Date: Wed, 09 Jul 2025 13:36:11 GMT
• Title: PBCAT: Patch-based composite adversarial training against physically realizable attacks on object detection
• Authors: Xiao Li, Yiming Zhu, Yifan Huang, Wei Zhang, Yingzhe He, Jie Shi, Xiaolin Hu,
• Abstract summary: Adversarial Training has been recognized as the most effective defense against adversarial attacks.<n>We propose PBCAT, a novel Patch-Based Composite Adversarial Training strategy.
• Score: 27.75925749085402
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Object detection plays a crucial role in many security-sensitive applications. However, several recent studies have shown that object detectors can be easily fooled by physically realizable attacks, \eg, adversarial patches and recent adversarial textures, which pose realistic and urgent threats. Adversarial Training (AT) has been recognized as the most effective defense against adversarial attacks. While AT has been extensively studied in the $l_\infty$ attack settings on classification models, AT against physically realizable attacks on object detectors has received limited exploration. Early attempts are only performed to defend against adversarial patches, leaving AT against a wider range of physically realizable attacks under-explored. In this work, we consider defending against various physically realizable attacks with a unified AT method. We propose PBCAT, a novel Patch-Based Composite Adversarial Training strategy. PBCAT optimizes the model by incorporating the combination of small-area gradient-guided adversarial patches and imperceptible global adversarial perturbations covering the entire image. With these designs, PBCAT has the potential to defend against not only adversarial patches but also unseen physically realizable attacks such as adversarial textures. Extensive experiments in multiple settings demonstrated that PBCAT significantly improved robustness against various physically realizable attacks over state-of-the-art defense methods. Notably, it improved the detection accuracy by 29.7\% over previous defense methods under one recent adversarial texture attack.

Related papers

• Meta Invariance Defense Towards Generalizable Robustness to Unknown Adversarial Attacks [62.036798488144306]
  Current defense mainly focuses on the known attacks, but the adversarial robustness to the unknown attacks is seriously overlooked. We propose an attack-agnostic defense method named Meta Invariance Defense (MID) We show that MID simultaneously achieves robustness to the imperceptible adversarial perturbations in high-level image classification and attack-suppression in low-level robust image regeneration.
  arXiv  Detail & Related papers  (2024-04-04T10:10:38Z)
• Fight Fire with Fire: Combating Adversarial Patch Attacks using Pattern-randomized Defensive Patches [12.329244399788669]
  Object detection is susceptible to adversarial patch attacks. In this paper, we propose a novel and general methodology for defending adversarial attacks. Two types of defensive patches, canary and woodpecker, are specially-crafted and injected into the model input to proactively probe or counteract potential adversarial patches.
  arXiv  Detail & Related papers  (2023-11-10T15:36:57Z)
• CBA: Contextual Background Attack against Optical Aerial Detection in the Physical World [8.826711009649133]
  Patch-based physical attacks have increasingly aroused concerns. Most existing methods focus on obscuring targets captured on the ground, and some of these methods are simply extended to deceive aerial detectors. We propose Contextual Background Attack (CBA), a novel physical attack framework against aerial detection, which can achieve strong attack efficacy and transferability in the physical world even without smudging the interested objects at all.
  arXiv  Detail & Related papers  (2023-02-27T05:10:27Z)
• Guidance Through Surrogate: Towards a Generic Diagnostic Attack [101.36906370355435]
  We develop a guided mechanism to avoid local minima during attack optimization, leading to a novel attack dubbed Guided Projected Gradient Attack (G-PGA) Our modified attack does not require random restarts, large number of attack iterations or search for an optimal step-size. More than an effective attack, G-PGA can be used as a diagnostic tool to reveal elusive robustness due to gradient masking in adversarial defenses.
  arXiv  Detail & Related papers  (2022-12-30T18:45:23Z)
• Benchmarking Adversarial Patch Against Aerial Detection [11.591143898488312]
  A novel adaptive-patch-based physical attack (AP-PA) framework is proposed. AP-PA generates adversarial patches that are adaptive in both physical dynamics and varying scales. We establish one of the first comprehensive, coherent, and rigorous benchmarks to evaluate the attack efficacy of adversarial patches on aerial detection tasks.
  arXiv  Detail & Related papers  (2022-10-30T07:55:59Z)
• Illusory Attacks: Information-Theoretic Detectability Matters in Adversarial Attacks [76.35478518372692]
  We introduce epsilon-illusory, a novel form of adversarial attack on sequential decision-makers. Compared to existing attacks, we empirically find epsilon-illusory to be significantly harder to detect with automated methods. Our findings suggest the need for better anomaly detectors, as well as effective hardware- and system-level defenses.
  arXiv  Detail & Related papers  (2022-07-20T19:49:09Z)
• Defending Against Person Hiding Adversarial Patch Attack with a Universal White Frame [28.128458352103543]
  High-performance object detection networks are vulnerable to adversarial patch attacks. Person-hiding attacks are emerging as a serious problem in many safety-critical applications. We propose a novel defense strategy that mitigates a person-hiding attack by optimizing defense patterns.
  arXiv  Detail & Related papers  (2022-04-27T15:18:08Z)
• Segment and Complete: Defending Object Detectors against Adversarial Patch Attacks with Robust Patch Detection [142.24869736769432]
  Adversarial patch attacks pose a serious threat to state-of-the-art object detectors. We propose Segment and Complete defense (SAC), a framework for defending object detectors against patch attacks. We show SAC can significantly reduce the targeted attack success rate of physical patch attacks.
  arXiv  Detail & Related papers  (2021-12-08T19:18:48Z)
• Evaluating the Robustness of Semantic Segmentation for Autonomous Driving against Real-World Adversarial Patch Attacks [62.87459235819762]
  In a real-world scenario like autonomous driving, more attention should be devoted to real-world adversarial examples (RWAEs) This paper presents an in-depth evaluation of the robustness of popular SS models by testing the effects of both digital and real-world adversarial patches.
  arXiv  Detail & Related papers  (2021-08-13T11:49:09Z)
• Robustness Out of the Box: Compositional Representations Naturally Defend Against Black-Box Patch Attacks [11.429509031463892]
  Patch-based adversarial attacks introduce a perceptible but localized change to the input that induces misclassification. In this work, we study two different approaches for defending against black-box patch attacks. We find that adversarial training has limited effectiveness against state-of-the-art location-optimized patch attacks.
  arXiv  Detail & Related papers  (2020-12-01T15:04:23Z)
• Adversarial Training against Location-Optimized Adversarial Patches [84.96938953835249]
  adversarial patches: clearly visible, but adversarially crafted rectangular patches in images. We first devise a practical approach to obtain adversarial patches while actively optimizing their location within the image. We apply adversarial training on these location-optimized adversarial patches and demonstrate significantly improved robustness on CIFAR10 and GTSRB.
  arXiv  Detail & Related papers  (2020-05-05T16:17:00Z)
• Certified Defenses for Adversarial Patches [72.65524549598126]
  Adversarial patch attacks are among the most practical threat models against real-world computer vision systems. This paper studies certified and empirical defenses against patch attacks.
  arXiv  Detail & Related papers  (2020-03-14T19:57:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.