Towards Assurance of LLM Adversarial Robustness using Ontology-Driven Argumentation

• URL: http://arxiv.org/abs/2410.07962v1
• Date: Thu, 10 Oct 2024 14:24:43 GMT
• Title: Towards Assurance of LLM Adversarial Robustness using Ontology-Driven Argumentation
• Authors: Tomas Bueno Momcilovic, Beat Buesser, Giulio Zizzo, Mark Purcell, Tomas Bueno Momcilovic,
• Abstract summary: We introduce a novel approach for assurance of large language models (LLMs) based on formal argumentation. We structure state-of-the-art attacks and defenses, facilitating creation of a human-readable assurance case. We provide implications for theory and practice, by targeting engineers, data scientists, users and auditors.
• Score: 1.368472250332885
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Despite the impressive adaptability of large language models (LLMs), challenges remain in ensuring their security, transparency, and interpretability. Given their susceptibility to adversarial attacks, LLMs need to be defended with an evolving combination of adversarial training and guardrails. However, managing the implicit and heterogeneous knowledge for continuously assuring robustness is difficult. We introduce a novel approach for assurance of the adversarial robustness of LLMs based on formal argumentation. Using ontologies for formalization, we structure state-of-the-art attacks and defenses, facilitating the creation of a human-readable assurance case, and a machine-readable representation. We demonstrate its application with examples in English language and code translation tasks, and provide implications for theory and practice, by targeting engineers, data scientists, users, and auditors.

Related papers

• Jailbreaking and Mitigation of Vulnerabilities in Large Language Models [4.564507064383306]
  Large Language Models (LLMs) have transformed artificial intelligence by advancing natural language understanding and generation. Despite these advancements, LLMs have shown considerable vulnerabilities, particularly to prompt injection and jailbreaking attacks. This review analyzes the state of research on these vulnerabilities and presents available defense strategies.
  arXiv  Detail & Related papers  (2024-10-20T00:00:56Z)
• Compromising Embodied Agents with Contextual Backdoor Attacks [69.71630408822767]
  Large language models (LLMs) have transformed the development of embodied intelligence. This paper uncovers a significant backdoor security threat within this process. By poisoning just a few contextual demonstrations, attackers can covertly compromise the contextual environment of a black-box LLM.
  arXiv  Detail & Related papers  (2024-08-06T01:20:12Z)
• Detecting and Understanding Vulnerabilities in Language Models via Mechanistic Interpretability [44.99833362998488]
  Large Language Models (LLMs) have shown impressive performance across a wide range of tasks. LLMs in particular are known to be vulnerable to adversarial attacks, where an imperceptible change to the input can mislead the output of the model. We propose a method, based on Mechanistic Interpretability (MI) techniques, to guide this process.
  arXiv  Detail & Related papers  (2024-07-29T09:55:34Z)
• Counterfactual Explainable Incremental Prompt Attack Analysis on Large Language Models [32.03992137755351]
  This study sheds light on the imperative need to bolster safety and privacy measures in large language models (LLMs) We propose Counterfactual Explainable Incremental Prompt Attack (CEIPA), a novel technique where we guide prompts in a specific manner to quantitatively measure attack effectiveness.
  arXiv  Detail & Related papers  (2024-07-12T14:26:14Z)
• Assessing Adversarial Robustness of Large Language Models: An Empirical Study [24.271839264950387]
  Large Language Models (LLMs) have revolutionized natural language processing, but their robustness against adversarial attacks remains a critical concern. We present a novel white-box style attack approach that exposes vulnerabilities in leading open-source LLMs, including Llama, OPT, and T5.
  arXiv  Detail & Related papers  (2024-05-04T22:00:28Z)
• Breaking Down the Defenses: A Comparative Survey of Attacks on Large Language Models [18.624280305864804]
  Large Language Models (LLMs) have become a cornerstone in the field of Natural Language Processing (NLP) This paper presents a comprehensive survey of the various forms of attacks targeting LLMs. We delve into topics such as adversarial attacks that aim to manipulate model outputs, data poisoning that affects model training, and privacy concerns related to training data exploitation.
  arXiv  Detail & Related papers  (2024-03-03T04:46:21Z)
• Data Poisoning for In-context Learning [49.77204165250528]
  In-context learning (ICL) has been recognized for its innovative ability to adapt to new tasks. This paper delves into the critical issue of ICL's susceptibility to data poisoning attacks. We introduce ICLPoison, a specialized attacking framework conceived to exploit the learning mechanisms of ICL.
  arXiv  Detail & Related papers  (2024-02-03T14:20:20Z)
• Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reasoning to Language Agents [80.5213198675411]
  Large language models (LLMs) have dramatically enhanced the field of language intelligence. LLMs leverage the intriguing chain-of-thought (CoT) reasoning techniques, obliging them to formulate intermediate steps en route to deriving an answer. Recent research endeavors have extended CoT reasoning methodologies to nurture the development of autonomous language agents.
  arXiv  Detail & Related papers  (2023-11-20T14:30:55Z)
• Visual Adversarial Examples Jailbreak Aligned Large Language Models [66.53468356460365]
  We show that the continuous and high-dimensional nature of the visual input makes it a weak link against adversarial attacks. We exploit visual adversarial examples to circumvent the safety guardrail of aligned LLMs with integrated vision. Our study underscores the escalating adversarial risks associated with the pursuit of multimodality.
  arXiv  Detail & Related papers  (2023-06-22T22:13:03Z)
• Red Teaming Language Model Detectors with Language Models [114.36392560711022]
  Large language models (LLMs) present significant safety and ethical risks if exploited by malicious users. Recent works have proposed algorithms to detect LLM-generated text and protect LLMs. We study two types of attack strategies: 1) replacing certain words in an LLM's output with their synonyms given the context; 2) automatically searching for an instructional prompt to alter the writing style of the generation.
  arXiv  Detail & Related papers  (2023-05-31T10:08:37Z)
• Semantic-Preserving Adversarial Code Comprehension [75.76118224437974]
  We propose Semantic-Preserving Adversarial Code Embeddings (SPACE) to find the worst-case semantic-preserving attacks. Experiments and analysis demonstrate that SPACE can stay robust against state-of-the-art attacks while boosting the performance of PrLMs for code.
  arXiv  Detail & Related papers  (2022-09-12T10:32:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.