Familiarity-Based Open-Set Recognition Under Adversarial Attacks

• URL: http://arxiv.org/abs/2311.05006v2
• Date: Thu, 02 Jan 2025 18:10:18 GMT
• Title: Familiarity-Based Open-Set Recognition Under Adversarial Attacks
• Authors: Philip Enevoldsen, Christian Gundersen, Nico Lang, Serge Belongie, Christian Igel,
• Abstract summary: We study gradient-based adversarial attacks on familiarity scores for both types of attacks, False Familiarity and False Novelty attacks. We formulate the adversarial reaction score as an alternative OSR scoring rule, which shows a high correlation with the MLS familiarity score.
• Score: 9.934489379453812
• License:
• Abstract: Open-set recognition (OSR), the identification of novel categories, can be a critical component when deploying classification models in real-world applications. Recent work has shown that familiarity-based scoring rules such as the Maximum Softmax Probability (MSP) or the Maximum Logit Score (MLS) are strong baselines when the closed-set accuracy is high. However, one of the potential weaknesses of familiarity-based OSR are adversarial attacks. Here, we study gradient-based adversarial attacks on familiarity scores for both types of attacks, False Familiarity and False Novelty attacks, and evaluate their effectiveness in informed and uninformed settings on TinyImageNet. Furthermore, we explore how novel and familiar samples react to adversarial attacks and formulate the adversarial reaction score as an alternative OSR scoring rule, which shows a high correlation with the MLS familiarity score.

Related papers

• AdvQDet: Detecting Query-Based Adversarial Attacks with Adversarial Contrastive Prompt Tuning [93.77763753231338]
  Adversarial Contrastive Prompt Tuning (ACPT) is proposed to fine-tune the CLIP image encoder to extract similar embeddings for any two intermediate adversarial queries. We show that ACPT can detect 7 state-of-the-art query-based attacks with $>99%$ detection rate within 5 shots. We also show that ACPT is robust to 3 types of adaptive attacks.
  arXiv  Detail & Related papers  (2024-08-04T09:53:50Z)
• PRAT: PRofiling Adversarial aTtacks [52.693011665938734]
  We introduce a novel problem of PRofiling Adversarial aTtacks (PRAT) Given an adversarial example, the objective of PRAT is to identify the attack used to generate it. We use AID to devise a novel framework for the PRAT objective.
  arXiv  Detail & Related papers  (2023-09-20T07:42:51Z)
• CARBEN: Composite Adversarial Robustness Benchmark [70.05004034081377]
  This paper demonstrates how composite adversarial attack (CAA) affects the resulting image. It provides real-time inferences of different models, which will facilitate users' configuration of the parameters of the attack level. A leaderboard to benchmark adversarial robustness against CAA is also introduced.
  arXiv  Detail & Related papers  (2022-07-16T01:08:44Z)
• On Trace of PGD-Like Adversarial Attacks [77.75152218980605]
  Adversarial attacks pose safety and security concerns for deep learning applications. We construct Adrial Response Characteristics (ARC) features to reflect the model's gradient consistency. Our method is intuitive, light-weighted, non-intrusive, and data-undemanding.
  arXiv  Detail & Related papers  (2022-05-19T14:26:50Z)
• Evaluation of Neural Networks Defenses and Attacks using NDCG and Reciprocal Rank Metrics [6.6389732792316]
  We present two metrics which are specifically designed to measure the effect of attacks, or the recovery effect of defenses, on the output of neural networks in classification tasks. Inspired by the normalized discounted cumulative gain and the reciprocal rank metrics used in information retrieval literature, we treat the neural network predictions as ranked lists of results. Compared to the common classification metrics, our proposed metrics demonstrate superior informativeness and distinctiveness.
  arXiv  Detail & Related papers  (2022-01-10T12:54:45Z)
• Membership Inference Attacks From First Principles [24.10746844866869]
  A membership inference attack allows an adversary to query a trained machine learning model to predict whether or not a particular example was contained in the model's training dataset. These attacks are currently evaluated using average-case "accuracy" metrics that fail to characterize whether the attack can confidently identify any members of the training set. We argue that attacks should instead be evaluated by computing their true-positive rate at low false-positive rates, and find most prior attacks perform poorly when evaluated in this way. Our attack is 10x more powerful at low false-positive rates, and also strictly dominates prior attacks on existing metrics.
  arXiv  Detail & Related papers  (2021-12-07T08:47:00Z)
• Towards A Conceptually Simple Defensive Approach for Few-shot classifiers Against Adversarial Support Samples [107.38834819682315]
  We study a conceptually simple approach to defend few-shot classifiers against adversarial attacks. We propose a simple attack-agnostic detection method, using the concept of self-similarity and filtering. Our evaluation on the miniImagenet (MI) and CUB datasets exhibit good attack detection performance.
  arXiv  Detail & Related papers  (2021-10-24T05:46:03Z)
• Adversarially Robust Classification based on GLRT [26.44693169694826]
  We show a defense strategy based on the generalized likelihood ratio test (GLRT), which jointly estimates the class of interest and the adversarial perturbation. We show that the GLRT approach yields performance competitive with that of the minimax approach under the worst-case attack. We also observe that the GLRT defense generalizes naturally to more complex models for which optimal minimax classifiers are not known.
  arXiv  Detail & Related papers  (2020-11-16T10:16:05Z)
• Unknown Presentation Attack Detection against Rational Attackers [6.351869353952288]
  Presentation attack detection and multimedia forensics are still vulnerable to attacks in real-life settings. Some of the challenges for existing solutions are the detection of unknown attacks, the ability to perform in adversarial settings, few-shot learning, and explainability. New optimization criterion is proposed and a set of requirements are defined for improving the performance of these systems in real-life settings.
  arXiv  Detail & Related papers  (2020-10-04T14:37:10Z)
• Detecting Adversarial Examples for Speech Recognition via Uncertainty Quantification [21.582072216282725]
  Machine learning systems and, specifically, automatic speech recognition (ASR) systems are vulnerable to adversarial attacks. In this paper, we focus on hybrid ASR systems and compare four acoustic models regarding their ability to indicate uncertainty under attack. We are able to detect adversarial examples with an area under the receiving operator curve score of more than 0.99.
  arXiv  Detail & Related papers  (2020-05-24T19:31:02Z)
• Temporal Sparse Adversarial Attack on Sequence-based Gait Recognition [56.844587127848854]
  We demonstrate that the state-of-the-art gait recognition model is vulnerable to such attacks. We employ a generative adversarial network based architecture to semantically generate adversarial high-quality gait silhouettes or video frames. The experimental results show that if only one-fortieth of the frames are attacked, the accuracy of the target model drops dramatically.
  arXiv  Detail & Related papers  (2020-02-22T10:08:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.