Adversarial Negotiation Dynamics in Generative Language Models

• URL: http://arxiv.org/abs/2501.00069v1
• Date: Sun, 29 Dec 2024 18:17:55 GMT
• Title: Adversarial Negotiation Dynamics in Generative Language Models
• Authors: Arinbjörn Kolbeinsson, Benedikt Kolbeinsson,
• Abstract summary: Generative language models are increasingly used for contract drafting and enhancement. This creates a scenario where competing parties deploy different language models against each other. We evaluate the performance and vulnerabilities of major open-source language models in head-to-head competitions.
• Score: 1.307537039737708
• License:
• Abstract: Generative language models are increasingly used for contract drafting and enhancement, creating a scenario where competing parties deploy different language models against each other. This introduces not only a game-theory challenge but also significant concerns related to AI safety and security, as the language model employed by the opposing party can be unknown. These competitive interactions can be seen as adversarial testing grounds, where models are effectively red-teamed to expose vulnerabilities such as generating biased, harmful or legally problematic text. Despite the importance of these challenges, the competitive robustness and safety of these models in adversarial settings remain poorly understood. In this small study, we approach this problem by evaluating the performance and vulnerabilities of major open-source language models in head-to-head competitions, simulating real-world contract negotiations. We further explore how these adversarial interactions can reveal potential risks, informing the development of more secure and reliable models. Our findings contribute to the growing body of research on AI safety, offering insights into model selection and optimisation in competitive legal contexts and providing actionable strategies for mitigating risks.

Related papers

• Safety at Scale: A Comprehensive Survey of Large Model Safety [299.801463557549]
  We present a comprehensive taxonomy of safety threats to large models, including adversarial attacks, data poisoning, backdoor attacks, jailbreak and prompt injection attacks, energy-latency attacks, data and model extraction attacks, and emerging agent-specific threats. We identify and discuss the open challenges in large model safety, emphasizing the need for comprehensive safety evaluations, scalable and effective defense mechanisms, and sustainable data practices.
  arXiv  Detail & Related papers  (2025-02-02T05:14:22Z)
• MultiAgent Collaboration Attack: Investigating Adversarial Attacks in Large Language Model Collaborations via Debate [24.92465108034783]
  Large Language Models (LLMs) have shown exceptional results on current benchmarks when working individually. The advancement in their capabilities, along with a reduction in parameter size and inference times, has facilitated the use of these models as agents. We evaluate the behavior of a network of models collaborating through debate under the influence of an adversary.
  arXiv  Detail & Related papers  (2024-06-20T20:09:37Z)
• Adversarial Attacks and Defense for Conversation Entailment Task [0.49157446832511503]
  Large language models are vulnerable to low-cost adversarial attacks. We fine-tune a transformer model to accurately discern the truthfulness of hypotheses. We introduce an embedding perturbation loss method to bolster the model's robustness.
  arXiv  Detail & Related papers  (2024-05-01T02:49:18Z)
• DPP-Based Adversarial Prompt Searching for Lanugage Models [56.73828162194457]
  Auto-regressive Selective Replacement Ascent (ASRA) is a discrete optimization algorithm that selects prompts based on both quality and similarity with determinantal point process (DPP) Experimental results on six different pre-trained language models demonstrate the efficacy of ASRA for eliciting toxic content.
  arXiv  Detail & Related papers  (2024-03-01T05:28:06Z)
• SA-Attack: Improving Adversarial Transferability of Vision-Language Pre-training Models via Self-Augmentation [56.622250514119294]
  In contrast to white-box adversarial attacks, transfer attacks are more reflective of real-world scenarios. We propose a self-augment-based transfer attack method, termed SA-Attack.
  arXiv  Detail & Related papers  (2023-12-08T09:08:50Z)
• Exploiting Large Language Models (LLMs) through Deception Techniques and Persuasion Principles [2.134057414078079]
  Large Language Models (LLMs) gain widespread use, ensuring their security and robustness is critical. This paper presents a novel study focusing on exploitation of such large language models against deceptive interactions. Our results demonstrate a significant finding in that these large language models are susceptible to deception and social engineering attacks.
  arXiv  Detail & Related papers  (2023-11-24T23:57:44Z)
• Robust Safety Classifier for Large Language Models: Adversarial Prompt Shield [7.5520641322945785]
  Large Language Models' safety remains a critical concern due to their vulnerability to adversarial attacks. We introduce the Adversarial Prompt Shield (APS), a lightweight model that excels in detection accuracy and demonstrates resilience against adversarial prompts. We also propose novel strategies for autonomously generating adversarial training datasets.
  arXiv  Detail & Related papers  (2023-10-31T22:22:10Z)
• A Survey on Transferability of Adversarial Examples across Deep Neural Networks [53.04734042366312]
  adversarial examples can manipulate machine learning models into making erroneous predictions. The transferability of adversarial examples enables black-box attacks which circumvent the need for detailed knowledge of the target model. This survey explores the landscape of the adversarial transferability of adversarial examples.
  arXiv  Detail & Related papers  (2023-10-26T17:45:26Z)
• Designing an attack-defense game: how to increase robustness of financial transaction models via a competition [69.08339915577206]
  Given the escalating risks of malicious attacks in the finance sector, understanding adversarial strategies and robust defense mechanisms for machine learning models is critical. We aim to investigate the current state and dynamics of adversarial attacks and defenses for neural network models that use sequential financial data as the input. We have designed a competition that allows realistic and detailed investigation of problems in modern financial transaction data. The participants compete directly against each other, so possible attacks and defenses are examined in close-to-real-life conditions.
  arXiv  Detail & Related papers  (2023-08-22T12:53:09Z)
• Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Models [11.323961700172175]
  This article investigates the challenges and risks associated with biases in large-scale language models like ChatGPT. We discuss the origins of biases, stemming from, among others, the nature of training data, model specifications, algorithmic constraints, product design, and policy decisions. We review the current approaches to identify, quantify, and mitigate biases in language models, emphasizing the need for a multi-disciplinary, collaborative effort to develop more equitable, transparent, and responsible AI systems.
  arXiv  Detail & Related papers  (2023-04-07T17:14:00Z)
• Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey [50.58063811745676]
  This work provides a survey of practical methods for addressing potential threats and societal harms from language generation models. We draw on several prior works' of language model risks to present a structured overview of strategies for detecting and ameliorating different kinds of risks/harms of language generators.
  arXiv  Detail & Related papers  (2022-10-14T10:43:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.