Enhancing Object Detection Robustness: Detecting and Restoring Confidence in the Presence of Adversarial Patch Attacks

• URL: http://arxiv.org/abs/2403.12988v2
• Date: Fri, 27 Jun 2025 13:45:14 GMT
• Title: Enhancing Object Detection Robustness: Detecting and Restoring Confidence in the Presence of Adversarial Patch Attacks
• Authors: Roie Kazoom, Raz Birman, Ofer Hadar,
• Abstract summary: This study evaluates defense mechanisms for the YOLOv5 model against adversarial patches.<n>We tested several defenses, including Segment and Complete (SAC), Inpainting, and Latent Diffusion Models.<n>Results indicate that adversarial patches reduce average detection confidence by 22.06%.
• Score: 2.963101656293054
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The widespread adoption of computer vision systems has underscored their susceptibility to adversarial attacks, particularly adversarial patch attacks on object detectors. This study evaluates defense mechanisms for the YOLOv5 model against such attacks. Optimized adversarial patches were generated and placed in sensitive image regions, by applying EigenCAM and grid search to determine optimal placement. We tested several defenses, including Segment and Complete (SAC), Inpainting, and Latent Diffusion Models. Our pipeline comprises three main stages: patch application, object detection, and defense analysis. Results indicate that adversarial patches reduce average detection confidence by 22.06\%. Defenses restored confidence levels by 3.45\% (SAC), 5.05\% (Inpainting), and significantly improved them by 26.61\%, which even exceeds the original accuracy levels, when using the Latent Diffusion Model, highlighting its superior effectiveness in mitigating the effects of adversarial patches.

Related papers

• Unleashing the Power of Pre-trained Encoders for Universal Adversarial Attack Detection [21.03032944637112]
  Adrial attacks pose a critical security threat to real-world AI systems. This paper proposes a lightweight adversarial detection framework based on the large-scale pre-trained vision-language model CLIP.
  arXiv  Detail & Related papers  (2025-04-01T05:21:45Z)
• A Hybrid Defense Strategy for Boosting Adversarial Robustness in Vision-Language Models [9.304845676825584]
  We propose a novel adversarial training framework that integrates multiple attack strategies and advanced machine learning techniques. Experiments conducted on real-world datasets, including CIFAR-10 and CIFAR-100, demonstrate that the proposed method significantly enhances model robustness.
  arXiv  Detail & Related papers  (2024-10-18T23:47:46Z)
• Robust Image Classification: Defensive Strategies against FGSM and PGD Adversarial Attacks [0.0]
  Adversarial attacks pose significant threats to the robustness of deep learning models in image classification. This paper explores and refines defense mechanisms against these attacks to enhance the resilience of neural networks.
  arXiv  Detail & Related papers  (2024-08-20T02:00:02Z)
• MirrorCheck: Efficient Adversarial Defense for Vision-Language Models [55.73581212134293]
  We propose a novel, yet elegantly simple approach for detecting adversarial samples in Vision-Language Models. Our method leverages Text-to-Image (T2I) models to generate images based on captions produced by target VLMs. Empirical evaluations conducted on different datasets validate the efficacy of our approach.
  arXiv  Detail & Related papers  (2024-06-13T15:55:04Z)
• Anomaly Unveiled: Securing Image Classification against Adversarial Patch Attacks [3.6275442368775512]
  Adversarial patch attacks pose a significant threat to the practical deployment of deep learning systems. In this paper, we investigate the behavior of adversarial patches as anomalies within the distribution of image information. Our proposed defense mechanism utilizes a clustering-based technique called DBSCAN to isolate anomalous image segments.
  arXiv  Detail & Related papers  (2024-02-09T08:52:47Z)
• DOEPatch: Dynamically Optimized Ensemble Model for Adversarial Patches Generation [12.995762461474856]
  We introduce the concept of energy and treat the adversarial patches generation process as an optimization of the adversarial patches to minimize the total energy of the person'' category. By adopting adversarial training, we construct a dynamically optimized ensemble model. We carried out six sets of comparative experiments and tested our algorithm on five mainstream object detection models.
  arXiv  Detail & Related papers  (2023-12-28T08:58:13Z)
• DefensiveDR: Defending against Adversarial Patches using Dimensionality Reduction [4.4100683691177816]
  Adrial patch-based attacks have shown to be a major deterrent towards the reliable use of machine learning models. We propose textitDefensiveDR, a practical mechanism using a dimensionality reduction technique to thwart such patch-based attacks.
  arXiv  Detail & Related papers  (2023-11-20T22:01:31Z)
• ODDR: Outlier Detection & Dimension Reduction Based Defense Against Adversarial Patches [4.4100683691177816]
  Adversarial attacks present a significant challenge to the dependable deployment of machine learning models. We propose Outlier Detection and Dimension Reduction (ODDR), a comprehensive defense strategy to counteract patch-based adversarial attacks. Our approach is based on the observation that input features corresponding to adversarial patches can be identified as outliers.
  arXiv  Detail & Related papers  (2023-11-20T11:08:06Z)
• LEAT: Towards Robust Deepfake Disruption in Real-World Scenarios via Latent Ensemble Attack [11.764601181046496]
  Deepfakes, malicious visual contents created by generative models, pose an increasingly harmful threat to society. To proactively mitigate deepfake damages, recent studies have employed adversarial perturbation to disrupt deepfake model outputs. We propose a simple yet effective disruption method called Latent Ensemble ATtack (LEAT), which attacks the independent latent encoding process.
  arXiv  Detail & Related papers  (2023-07-04T07:00:37Z)
• Benchmarking Adversarial Patch Against Aerial Detection [11.591143898488312]
  A novel adaptive-patch-based physical attack (AP-PA) framework is proposed. AP-PA generates adversarial patches that are adaptive in both physical dynamics and varying scales. We establish one of the first comprehensive, coherent, and rigorous benchmarks to evaluate the attack efficacy of adversarial patches on aerial detection tasks.
  arXiv  Detail & Related papers  (2022-10-30T07:55:59Z)
• Adv-Attribute: Inconspicuous and Transferable Adversarial Attack on Face Recognition [111.1952945740271]
  Adversarial Attributes (Adv-Attribute) is designed to generate inconspicuous and transferable attacks on face recognition. Experiments on the FFHQ and CelebA-HQ datasets show that the proposed Adv-Attribute method achieves the state-of-the-art attacking success rates.
  arXiv  Detail & Related papers  (2022-10-13T09:56:36Z)
• ObjectSeeker: Certifiably Robust Object Detection against Patch Hiding Attacks via Patch-agnostic Masking [95.6347501381882]
  Object detectors are found to be vulnerable to physical-world patch hiding attacks. We propose ObjectSeeker as a framework for building certifiably robust object detectors.
  arXiv  Detail & Related papers  (2022-02-03T19:34:25Z)
• Segment and Complete: Defending Object Detectors against Adversarial Patch Attacks with Robust Patch Detection [142.24869736769432]
  Adversarial patch attacks pose a serious threat to state-of-the-art object detectors. We propose Segment and Complete defense (SAC), a framework for defending object detectors against patch attacks. We show SAC can significantly reduce the targeted attack success rate of physical patch attacks.
  arXiv  Detail & Related papers  (2021-12-08T19:18:48Z)
• Model-Agnostic Meta-Attack: Towards Reliable Evaluation of Adversarial Robustness [53.094682754683255]
  We propose a Model-Agnostic Meta-Attack (MAMA) approach to discover stronger attack algorithms automatically. Our method learns the in adversarial attacks parameterized by a recurrent neural network. We develop a model-agnostic training algorithm to improve the ability of the learned when attacking unseen defenses.
  arXiv  Detail & Related papers  (2021-10-13T13:54:24Z)
• Adaptive Feature Alignment for Adversarial Training [56.17654691470554]
  CNNs are typically vulnerable to adversarial attacks, which pose a threat to security-sensitive applications. We propose the adaptive feature alignment (AFA) to generate features of arbitrary attacking strengths. Our method is trained to automatically align features of arbitrary attacking strength.
  arXiv  Detail & Related papers  (2021-05-31T17:01:05Z)
• Towards Adversarial Patch Analysis and Certified Defense against Crowd Counting [61.99564267735242]
  Crowd counting has drawn much attention due to its importance in safety-critical surveillance systems. Recent studies have demonstrated that deep neural network (DNN) methods are vulnerable to adversarial attacks. We propose a robust attack strategy called Adversarial Patch Attack with Momentum to evaluate the robustness of crowd counting models.
  arXiv  Detail & Related papers  (2021-04-22T05:10:55Z)
• How Robust are Randomized Smoothing based Defenses to Data Poisoning? [66.80663779176979]
  We present a previously unrecognized threat to robust machine learning models that highlights the importance of training-data quality. We propose a novel bilevel optimization-based data poisoning attack that degrades the robustness guarantees of certifiably robust classifiers. Our attack is effective even when the victim trains the models from scratch using state-of-the-art robust training methods.
  arXiv  Detail & Related papers  (2020-12-02T15:30:21Z)
• A Self-supervised Approach for Adversarial Robustness [105.88250594033053]
  Adversarial examples can cause catastrophic mistakes in Deep Neural Network (DNNs) based vision systems. This paper proposes a self-supervised adversarial training mechanism in the input space. It provides significant robustness against the textbfunseen adversarial attacks.
  arXiv  Detail & Related papers  (2020-06-08T20:42:39Z)
• Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks [65.20660287833537]
  In this paper we propose two extensions of the PGD-attack overcoming failures due to suboptimal step size and problems of the objective function. We then combine our novel attacks with two complementary existing ones to form a parameter-free, computationally affordable and user-independent ensemble of attacks to test adversarial robustness.
  arXiv  Detail & Related papers  (2020-03-03T18:15:55Z)
• (De)Randomized Smoothing for Certifiable Defense against Patch Attacks [136.79415677706612]
  We introduce a certifiable defense against patch attacks that guarantees for a given image and patch attack size. Our method is related to the broad class of randomized smoothing robustness schemes. Our results effectively establish a new state-of-the-art of certifiable defense against patch attacks on CIFAR-10 and ImageNet.
  arXiv  Detail & Related papers  (2020-02-25T08:39:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.