Experiments with Detecting and Mitigating AI Deception

• URL: http://arxiv.org/abs/2306.14816v1
• Date: Mon, 26 Jun 2023 16:22:13 GMT
• Title: Experiments with Detecting and Mitigating AI Deception
• Abstract summary: How to detect and deceptive AI systems is an open problem for the field of safe and trustworthy AI. We analyse two algorithms for mitigating deception: The first is based on the path-specific objectives framework where paths in the game that incentivise deception are removed. The second is based on shielding, i.e., monitoring for unsafe policies and replacing them with a safe reference policy.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: How to detect and mitigate deceptive AI systems is an open problem for the field of safe and trustworthy AI. We analyse two algorithms for mitigating deception: The first is based on the path-specific objectives framework where paths in the game that incentivise deception are removed. The second is based on shielding, i.e., monitoring for unsafe policies and replacing them with a safe reference policy. We construct two simple games and evaluate our algorithms empirically. We find that both methods ensure that our agent is not deceptive, however, shielding tends to achieve higher reward.

Related papers

• The Scenic Route to Deception: Dark Patterns and Explainability Pitfalls in Conversational Navigation [0.0]
  We argue that conversational interfaces introduce risks of manipulation and misplaced trust.<n>We categorise these risks using a 2x2 framework based on intent and origin.<n>We suggest that one robust way to operationalise trustworthy conversational navigation is through neuro-symbolic architecture.
  arXiv  Detail & Related papers  (2026-03-15T20:10:00Z)
• Trust Over Fear: How Motivation Framing in System Prompts Affects AI Agent Debugging Depth [0.0]
  We present two studies investigating how trust-based versus fear-based motivation framing in system prompts influences AI agent performance.<n>Trust-framed agents found 59% more hidden issues while taking 83% more investigative steps.<n>Fear-framed (PUA) agents showed no significant improvement over baseline on any metric.
  arXiv  Detail & Related papers  (2026-03-15T13:25:52Z)
• What Is AI Safety? What Do We Want It to Be? [0.0]
  A research project falls within the purview of AI safety just in case it aims to prevent or reduce the harms caused by AI systems.<n>Despite its simplicity and appeal, we argue that The Safety Conception is in tension with at least two trends in the ways AI safety researchers and organizations think and talk about AI safety.
  arXiv  Detail & Related papers  (2025-05-05T01:55:00Z)
• Among Us: A Sandbox for Agentic Deception [1.1893676124374688]
  Among Us is a text-based social-deduction game environment. LLM-agents exhibit human-style deception naturally while they think, speak, and act with other agents or humans. We evaluate the effectiveness of AI safety techniques for detecting lying and deception in Among Us.
  arXiv  Detail & Related papers  (2025-04-05T06:09:32Z)
• Deceptive Sequential Decision-Making via Regularized Policy Optimization [54.38738815697299]
  Two regularization strategies for policy synthesis problems that actively deceive an adversary about a system's underlying rewards are presented. We show how each form of deception can be implemented in policy optimization problems. We show that diversionary deception can cause the adversary to believe that the most important agent is the least important, while attaining a total accumulated reward that is $98.83%$ of its optimal, non-deceptive value.
  arXiv  Detail & Related papers  (2025-01-30T23:41:40Z)
• Illusions of Relevance: Using Content Injection Attacks to Deceive Retrievers, Rerankers, and LLM Judges [52.96987928118327]
  We find that embedding models for retrieval, rerankers, and large language model (LLM) relevance judges are vulnerable to content injection attacks. We identify two primary threats: (1) inserting unrelated or harmful content within passages that still appear deceptively "relevant", and (2) inserting entire queries or key query terms into passages to boost their perceived relevance. Our study systematically examines the factors that influence an attack's success, such as the placement of injected content and the balance between relevant and non-relevant material.
  arXiv  Detail & Related papers  (2025-01-30T18:02:15Z)
• Inception: Efficiently Computable Misinformation Attacks on Markov Games [14.491458698581038]
  We study security threats to Markov games due to information asymmetry and misinformation. We derive the victim's policy under worst-case rationality and present-time algorithms to compute the attacker's optimal worst-case policy. Our work exposes a security vulnerability from standard game assumptions under misinformation.
  arXiv  Detail & Related papers  (2024-06-24T20:01:43Z)
• Honesty Is the Best Policy: Defining and Mitigating AI Deception [26.267047631872366]
  We focus on the problem that agents might deceive in order to achieve their goals. We introduce a formal definition of deception in structural causal games. We show, experimentally, that these results can be used to mitigate deception in reinforcement learning agents and language models.
  arXiv  Detail & Related papers  (2023-12-03T11:11:57Z)
• Enhancing Infrared Small Target Detection Robustness with Bi-Level Adversarial Framework [61.34862133870934]
  We propose a bi-level adversarial framework to promote the robustness of detection in the presence of distinct corruptions. Our scheme remarkably improves 21.96% IOU across a wide array of corruptions and notably promotes 4.97% IOU on the general benchmark.
  arXiv  Detail & Related papers  (2023-09-03T06:35:07Z)
• System III: Learning with Domain Knowledge for Safety Constraints [0.0]
  This paper proposes a novel framework for incorporating domain knowledge to help guide safe exploration and boost sample efficiency. In our approach, called $textitSystem III$, we represent domain expert knowledge of safety in form of first-order logic. We evaluate the effectiveness of the proposed method on OpenAI's Gym and Safety-Gym environments.
  arXiv  Detail & Related papers  (2023-04-23T09:44:41Z)
• Approximate Shielding of Atari Agents for Safe Exploration [83.55437924143615]
  We propose a principled algorithm for safe exploration based on the concept of shielding. We present preliminary results that show our approximate shielding algorithm effectively reduces the rate of safety violations.
  arXiv  Detail & Related papers  (2023-04-21T16:19:54Z)
• Near-Optimal Multi-Agent Learning for Safe Coverage Control [76.99020416197631]
  In multi-agent coverage control problems, agents navigate their environment to reach locations that maximize the coverage of some density. In this paper, we aim to efficiently learn the density to approximately solve the coverage problem while preserving the agents' safety. We give first of its kind results: near optimal coverage in finite time while provably guaranteeing safety.
  arXiv  Detail & Related papers  (2022-10-12T16:33:34Z)
• Illusory Attacks: Information-Theoretic Detectability Matters in Adversarial Attacks [76.35478518372692]
  We introduce epsilon-illusory, a novel form of adversarial attack on sequential decision-makers. Compared to existing attacks, we empirically find epsilon-illusory to be significantly harder to detect with automated methods. Our findings suggest the need for better anomaly detectors, as well as effective hardware- and system-level defenses.
  arXiv  Detail & Related papers  (2022-07-20T19:49:09Z)
• Surveillance Evasion Through Bayesian Reinforcement Learning [78.79938727251594]
  We consider a 2D continuous path planning problem with a completely unknown intensity of random termination. Those Observers' surveillance intensity is a priori unknown and has to be learned through repetitive path planning.
  arXiv  Detail & Related papers  (2021-09-30T02:29:21Z)
• Exploration and Incentives in Reinforcement Learning [107.42240386544633]
  We consider complex exploration problems, where each agent faces the same (but unknown) MDP. Agents control the choice of policies, whereas an algorithm can only issue recommendations. We design an algorithm which explores all reachable states in the MDP.
  arXiv  Detail & Related papers  (2021-02-28T00:15:53Z)
• Discovering Imperfectly Observable Adversarial Actions using Anomaly Detection [0.24244694855867271]
  Anomaly detection is a method for discovering unusual and suspicious behavior. We propose two algorithms for solving such games. Experiments show that both algorithms are applicable for cases with low feature space dimensions.
  arXiv  Detail & Related papers  (2020-04-22T15:31:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.