Safeguards Based on Copyable Context Cannot Provide Reliable Safety for LLMs

• URL: http://arxiv.org/abs/2607.27951v1
• Date: Thu, 30 Jul 2026 09:59:08 GMT
• Title: Safeguards Based on Copyable Context Cannot Provide Reliable Safety for LLMs
• Abstract summary: Large language model safeguards decide whether to answer before seeing how an answer will be used.<n>This creates a basic problem for dual-use tasks: the same answer can help an authorized professional or an attacker, while an attacker can imitate a benign request and interaction history.<n>We separate the capability released by the model from the evidence available about downstream use. When that evidence is copyable, we derive the exact worst-case floor on attacker assistance while preserving useful answers.
• Score: 61.39153106347947
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Large language model safeguards decide whether to answer before seeing how an answer will be used. This creates a basic problem for dual-use tasks: the same answer can help an authorized professional or an attacker, while an attacker can imitate a benign request and interaction history. We separate the capability released by the model from the evidence available about downstream use. When that evidence is copyable, we derive the exact worst-case floor on attacker assistance while preserving useful answers. The result yields a safety trilemma: Useful Capability, Reliable Safety, and Open Access cannot coexist. We then show how a trusted credential can complement existing safeguards by adding hard-to-copy information that predicts actual downstream use, and identify the stronger condition needed to eliminate the floor. Evidence from dual-use evaluations, adaptive attacks, and deployed trusted-access programs supports the practical relevance of these conditions.

Related papers

• Adversarial Attacks for Good: A Survey of Proactive Protection across the Visual Content Lifecycle [65.6642776933861]
  This survey examines the protective paradigm that has grown around this intervention point.<n>We find that most protections are still validated mainly against static or weakly adaptive adversaries.<n>We close by consolidating cross-stage countermeasures and open problems for robust, composable, and deployable owner-side protection.
  arXiv  Detail & Related papers  (2026-08-05T00:46:50Z)
• Defeater Cards: Characterizing and Managing Safety Assurance Case Defeaters [4.510181955306307]
  Safety assurance cases provide structured justifications that safety-critical systems meet their safety requirements.<n>Recently, the notion of defeaters has emerged as a rigorous means of challenging the validity of safety arguments.<n>We propose Defeater Cards, a new structured documentation artifact for systematically characterizing, reasoning about, and managing defeaters in safety cases.
  arXiv  Detail & Related papers  (2026-06-09T21:33:07Z)
• AgentSecBench: Measuring Prompt Injection, Privacy Leakage, and Tool-Use Integrity in LLM Agents [0.2864713389096699]
  We introduce AgentSecBench as an empirical instantiation of a formal security framework for this problem.<n>It defines three games-instruction-integrity, retrieval-confidentiality, and capability-integrity-under a common notion of intent-to-execution noninterference with permitted leakage.<n>It represents an application policy as a projection onto authorized observations and capabilities, distinguishes prompt annotations from enforcing projections, and measures both adversarial advantage and whether a defense closes the relevant model-visible channel before generation.
  arXiv  Detail & Related papers  (2026-05-25T18:53:22Z)
• Membership Inference Attacks for Retrieval Based In-Context Learning for Document Question Answering [5.932575574212546]
  We show that remotely hosted applications employing in-context learning can be vulnerable to membership-inference attacks.<n>We propose two black-box membership inference attacks that exploit query text prefixes to distinguish member from non-member inputs.
  arXiv  Detail & Related papers  (2026-05-05T08:19:17Z)
• Evaluating Language Model Reasoning about Confidential Information [95.64687778185703]
  We study whether language models exhibit contextual robustness, or the capability to adhere to context-dependent safety specifications.<n>We develop a benchmark (PasswordEval) that measures whether language models can correctly determine when a user request is authorized.<n>We find that current open- and closed-source models struggle with this seemingly simple task, and that, perhaps surprisingly, reasoning capabilities do not generally improve performance.
  arXiv  Detail & Related papers  (2025-08-27T15:39:46Z)
• Benchmarking Misuse Mitigation Against Covert Adversaries [80.74502950627736]
  Existing language model safety evaluations focus on overt attacks and low-stakes tasks.<n>We develop Benchmarks for Stateful Defenses (BSD), a data generation pipeline that automates evaluations of covert attacks and corresponding defenses.<n>Our evaluations indicate that decomposition attacks are effective misuse enablers, and highlight stateful defenses as a countermeasure.
  arXiv  Detail & Related papers  (2025-06-06T17:33:33Z)
• Towards Copyright Protection for Knowledge Bases of Retrieval-augmented Language Models via Reasoning [58.57194301645823]
  Large language models (LLMs) are increasingly integrated into real-world personalized applications.<n>The valuable and often proprietary nature of the knowledge bases used in RAG introduces the risk of unauthorized usage by adversaries.<n>Existing methods that can be generalized as watermarking techniques to protect these knowledge bases typically involve poisoning or backdoor attacks.<n>We propose name for harmless' copyright protection of knowledge bases.
  arXiv  Detail & Related papers  (2025-02-10T09:15:56Z)
• Gandalf the Red: Adaptive Security for LLMs [2.9422902813085665]
  Current evaluations of defenses against prompt attacks in large language model (LLM) applications overlook two critical factors: the dynamic nature of adversarial behavior and the usability penalties imposed on legitimate users by restrictive defenses.<n>We propose D-SEC, which explicitly separates attackers from legitimate users, models multi-step interactions, and expresses the security-utility in an optimizable form.
  arXiv  Detail & Related papers  (2025-01-14T08:30:49Z)
• Certifying LLM Safety against Adversarial Prompting [70.96868018621167]
  Large language models (LLMs) are vulnerable to adversarial attacks that add malicious tokens to an input prompt.<n>We introduce erase-and-check, the first framework for defending against adversarial prompts with certifiable safety guarantees.
  arXiv  Detail & Related papers  (2023-09-06T04:37:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.