Enhancing Stateful Detection of Adversarial Attacks with Soft-labels' Temporality and Robust Similarity Approximations

• URL: http://arxiv.org/abs/2606.21592v1
• Date: Fri, 19 Jun 2026 16:49:29 GMT
• Title: Enhancing Stateful Detection of Adversarial Attacks with Soft-labels' Temporality and Robust Similarity Approximations
• Abstract summary: Stateful Detection (SD) mitigates adversarial attacks by determining whether a sequence of queries contains queries from a black-box adversary.<n>Recent works, such as Blacklight and PIHA utilize query similarity to detect such queries.<n>In this paper, we observe that temporal information, in particular, the temporal correlation of the classification soft labels, is a prominent characteristic of adversarial attacks.
• Score: 25.36313126508004
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Stateful Detection (SD) mitigates adversarial attacks by determining whether a sequence of queries contains queries from a black-box adversary. Recent works, such as Blacklight and PIHA utilize query similarity to detect such queries. In this paper, we observe that temporal information, in particular, the temporal correlation of the classification soft labels, is a prominent characteristic of adversarial attacks and can be leveraged to reduce false positive rates. Moreover, we point out a potential vulnerability in SD implementation. Many SD systems identify similar queries according to some implicit, computationally expensive metric. To improve efficiency, these systems often adopt an approximate similarity function as substitute. This discrepancy could be exploited by crafting queries that appear dissimilar under the approximation but are close in the intended metric, thereby evading detection. We refer to this as an ``adversarial attack'' on the approximation function, and demonstrate it through a lightweight attack on Blacklight's similarity function. Based on the above observations, we propose a two-phase approach. The first phase identifies subsequences of queries with high similarity, incorporating randomness to prevent the aforementioned ``adversarial attacks''. The second phase analyzes temporal correlation of the soft-labels to further validate the presence of the adversary's queries. Experimental results show that the framework detects adversarial queries generated by Boundary Attack, HSJA, SimBA, Square Attack with true positive rate (TPR) reaching 1.00, while maintaining a false positive rate (FPR) of at most 0.06. Additionally, the method is robust against OARS which is an adaptive attack.

Related papers

• Robust Context-Aware Detection of Malicious Instructions in Text [22.27527497326564]
  A fundamental task in addressing this vulnerability is successful segmentation of a given text into benign and malicious sentences.<n>We develop an approach for malicious sentence classification that is both context- and query-aware.<n>We show that the proposed approach outperforms state-of-the-art IPI defense baselines under static attacks.
  arXiv  Detail & Related papers  (2026-08-05T21:44:44Z)
• How Worst-Case Are Adversarial Attacks? Linking Adversarial and Perturbation Robustness [4.60092781176058]
  Adrial attacks are widely used to identify model vulnerabilities, but their validity as proxies for robustness to random perturbations remains debated.<n>We ask whether an adversarial example provides a representative estimate of misprediction risk under perturbations of the same magnitude.<n>We study the limits of this connection by proposing an attack strategy designed to probe vulnerabilities in regimes that are statistically closer to uniform noise.
  arXiv  Detail & Related papers  (2026-01-20T22:24:47Z)
• AdvQDet: Detecting Query-Based Adversarial Attacks with Adversarial Contrastive Prompt Tuning [93.77763753231338]
  Adversarial Contrastive Prompt Tuning (ACPT) is proposed to fine-tune the CLIP image encoder to extract similar embeddings for any two intermediate adversarial queries. We show that ACPT can detect 7 state-of-the-art query-based attacks with $>99%$ detection rate within 5 shots. We also show that ACPT is robust to 3 types of adaptive attacks.
  arXiv  Detail & Related papers  (2024-08-04T09:53:50Z)
• Query Provenance Analysis: Efficient and Robust Defense against Query-based Black-box Attacks [11.32992178606254]
  We propose a novel approach, Query Provenance Analysis (QPA), for more robust and efficient Stateful Defense Models (SDMs) QPA encapsulates the historical relationships among queries as the sequence feature to capture the fundamental difference between benign and adversarial query sequences. We evaluate QPA compared with two baselines, BlackLight and PIHA, on four widely used datasets with six query-based black-box attack algorithms.
  arXiv  Detail & Related papers  (2024-05-31T06:56:54Z)
• Mitigating LLM Hallucinations via Conformal Abstention [70.83870602967625]
  We develop a principled procedure for determining when a large language model should abstain from responding in a general domain. We leverage conformal prediction techniques to develop an abstention procedure that benefits from rigorous theoretical guarantees on the hallucination rate (error rate) Experimentally, our resulting conformal abstention method reliably bounds the hallucination rate on various closed-book, open-domain generative question answering datasets.
  arXiv  Detail & Related papers  (2024-04-04T11:32:03Z)
• Token-Level Adversarial Prompt Detection Based on Perplexity Measures and Contextual Information [67.78183175605761]
  Large Language Models are susceptible to adversarial prompt attacks. This vulnerability underscores a significant concern regarding the robustness and reliability of LLMs. We introduce a novel approach to detecting adversarial prompts at a token level.
  arXiv  Detail & Related papers  (2023-11-20T03:17:21Z)
• On the Universal Adversarial Perturbations for Efficient Data-free Adversarial Detection [55.73320979733527]
  We propose a data-agnostic adversarial detection framework, which induces different responses between normal and adversarial samples to UAPs. Experimental results show that our method achieves competitive detection performance on various text classification tasks.
  arXiv  Detail & Related papers  (2023-06-27T02:54:07Z)
• On Trace of PGD-Like Adversarial Attacks [77.75152218980605]
  Adversarial attacks pose safety and security concerns for deep learning applications. We construct Adrial Response Characteristics (ARC) features to reflect the model's gradient consistency. Our method is intuitive, light-weighted, non-intrusive, and data-undemanding.
  arXiv  Detail & Related papers  (2022-05-19T14:26:50Z)
• ADC: Adversarial attacks against object Detection that evade Context consistency checks [55.8459119462263]
  We show that even context consistency checks can be brittle to properly crafted adversarial examples. We propose an adaptive framework to generate examples that subvert such defenses. Our results suggest that how to robustly model context and check its consistency, is still an open problem.
  arXiv  Detail & Related papers  (2021-10-24T00:25:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.