Proactive Defense: Compound AI for Detecting Persuasion Attacks and Measuring Inoculation Effectiveness

• URL: http://arxiv.org/abs/2511.21749v1
• Date: Sun, 23 Nov 2025 07:49:05 GMT
• Title: Proactive Defense: Compound AI for Detecting Persuasion Attacks and Measuring Inoculation Effectiveness
• Authors: Svitlana Volkova, Will Dupree, Hsien-Te Kao, Peter Bautista, Gabe Ganberg, Jeff Beaubien, Laura Cassani,
• Abstract summary: This paper introduces BRIES, a novel compound AI architecture designed to detect and measure the effectiveness of persuasion attacks.<n>We present a system with specialized agents: a Twister that generates adversarial content employing targeted persuasion tactics, a Detector that identifies attack types with parameters, a Defender that creates resilient content through content inoculation, and an assessor that employs causal inference to evaluate inoculation effectiveness.
• Score: 1.1686584679011365
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper introduces BRIES, a novel compound AI architecture designed to detect and measure the effectiveness of persuasion attacks across information environments. We present a system with specialized agents: a Twister that generates adversarial content employing targeted persuasion tactics, a Detector that identifies attack types with configurable parameters, a Defender that creates resilient content through content inoculation, and an Assessor that employs causal inference to evaluate inoculation effectiveness. Experimenting with the SemEval 2023 Task 3 taxonomy across the synthetic persuasion dataset, we demonstrate significant variations in detection performance across language agents. Our comparative analysis reveals significant performance disparities with GPT-4 achieving superior detection accuracy on complex persuasion techniques, while open-source models like Llama3 and Mistral demonstrated notable weaknesses in identifying subtle rhetorical, suggesting that different architectures encode and process persuasive language patterns in fundamentally different ways. We show that prompt engineering dramatically affects detection efficacy, with temperature settings and confidence scoring producing model-specific variations; Gemma and GPT-4 perform optimally at lower temperatures while Llama3 and Mistral show improved capabilities at higher temperatures. Our causal analysis provides novel insights into socio-emotional-cognitive signatures of persuasion attacks, revealing that different attack types target specific cognitive dimensions. This research advances generative AI safety and cognitive security by quantifying LLM-specific vulnerabilities to persuasion attacks and delivers a framework for enhancing human cognitive resilience through structured interventions before exposure to harmful content.

Related papers

• Benchmarking Gaslighting Attacks Against Speech Large Language Models [31.842578503471586]
  We introduce gaslighting attacks, strategically crafted prompts designed to mislead, override, or distort model reasoning.<n>Specifically, we construct five manipulation strategies: Anger, Cognitive Disruption, Sarcasm, Implicit, and Professional Negation.<n>Our framework captures both performance degradation and behavioral responses, including unsolicited apologies and refusals.
  arXiv  Detail & Related papers  (2025-09-24T07:57:10Z)
• Deep Learning Models for Robust Facial Liveness Detection [56.08694048252482]
  This study introduces a robust solution through novel deep learning models addressing the deficiencies in contemporary anti-spoofing techniques.<n>By innovatively integrating texture analysis and reflective properties associated with genuine human traits, our models distinguish authentic presence from replicas with remarkable precision.
  arXiv  Detail & Related papers  (2025-08-12T17:19:20Z)
• A Survey on Model Extraction Attacks and Defenses for Large Language Models [55.60375624503877]
  Model extraction attacks pose significant security threats to deployed language models.<n>This survey provides a comprehensive taxonomy of extraction attacks and defenses, categorizing attacks into functionality extraction, training data extraction, and prompt-targeted attacks.<n>We examine defense mechanisms organized into model protection, data privacy protection, and prompt-targeted strategies, evaluating their effectiveness across different deployment scenarios.
  arXiv  Detail & Related papers  (2025-06-26T22:02:01Z)
• A Knowledge-guided Adversarial Defense for Resisting Malicious Visual Manipulation [93.28532038721816]
  Malicious applications of visual manipulation have raised serious threats to the security and reputation of users in many fields.<n>We propose a knowledge-guided adversarial defense (KGAD) to actively force malicious manipulation models to output semantically confusing samples.
  arXiv  Detail & Related papers  (2025-04-11T10:18:13Z)
• Countering Backdoor Attacks in Image Recognition: A Survey and Evaluation of Mitigation Strategies [10.801476967873173]
  We present a review of existing mitigation strategies designed to counter backdoor attacks in image recognition.<n>We conduct an extensive benchmarking of sixteen state-of-the-art approaches against eight distinct backdoor attacks.<n>Our results, derived from 122,236 individual experiments, indicate that while many approaches provide some level of protection, their performance can vary considerably.
  arXiv  Detail & Related papers  (2024-11-17T23:30:01Z)
• Systematically Analyzing Prompt Injection Vulnerabilities in Diverse LLM Architectures [5.062846614331549]
  This study systematically analyzes the vulnerability of 36 large language models (LLMs) to various prompt injection attacks. Across 144 prompt injection tests, we observed a strong correlation between model parameters and vulnerability.
  arXiv  Detail & Related papers  (2024-10-28T18:55:21Z)
• Counterfactual Explainable Incremental Prompt Attack Analysis on Large Language Models [32.03992137755351]
  This study sheds light on the imperative need to bolster safety and privacy measures in large language models (LLMs) We propose Counterfactual Explainable Incremental Prompt Attack (CEIPA), a novel technique where we guide prompts in a specific manner to quantitatively measure attack effectiveness.
  arXiv  Detail & Related papers  (2024-07-12T14:26:14Z)
• Investigating Human-Identifiable Features Hidden in Adversarial Perturbations [54.39726653562144]
  Our study explores up to five attack algorithms across three datasets. We identify human-identifiable features in adversarial perturbations. Using pixel-level annotations, we extract such features and demonstrate their ability to compromise target models.
  arXiv  Detail & Related papers  (2023-09-28T22:31:29Z)
• Avoid Adversarial Adaption in Federated Learning by Multi-Metric Investigations [55.2480439325792]
  Federated Learning (FL) facilitates decentralized machine learning model training, preserving data privacy, lowering communication costs, and boosting model performance through diversified data sources. FL faces vulnerabilities such as poisoning attacks, undermining model integrity with both untargeted performance degradation and targeted backdoor attacks. We define a new notion of strong adaptive adversaries, capable of adapting to multiple objectives simultaneously. MESAS is the first defense robust against strong adaptive adversaries, effective in real-world data scenarios, with an average overhead of just 24.37 seconds.
  arXiv  Detail & Related papers  (2023-06-06T11:44:42Z)
• Exploiting Explainability to Design Adversarial Attacks and Evaluate Attack Resilience in Hate-Speech Detection Models [0.47334880432883714]
  We present an analysis of adversarial robustness exhibited by various hate-speech detection models. We devise and execute targeted attacks on the text by leveraging the TextAttack tool. This work paves the way for creating more robust and reliable hate-speech detection systems.
  arXiv  Detail & Related papers  (2023-05-29T19:59:40Z)
• Towards Understanding the Adversarial Vulnerability of Skeleton-based Action Recognition [133.35968094967626]
  Skeleton-based action recognition has attracted increasing attention due to its strong adaptability to dynamic circumstances. With the help of deep learning techniques, it has also witnessed substantial progress and currently achieved around 90% accuracy in benign environment. Research on the vulnerability of skeleton-based action recognition under different adversarial settings remains scant.
  arXiv  Detail & Related papers  (2020-05-14T17:12:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.