Internal Safety Collapse in Frontier Large Language Models

• URL: http://arxiv.org/abs/2603.23509v1
• Date: Wed, 04 Mar 2026 12:55:34 GMT
• Title: Internal Safety Collapse in Frontier Large Language Models
• Abstract summary: This work identifies a critical failure mode in frontier large language models (LLMs)<n>Under certain task conditions, models enter a state in which they continuously generate harmful content while executing otherwise benign tasks.<n>We introduce TVD, a framework that triggers ISC through domain tasks where generating harmful content is the only valid completion.
• Score: 65.00730294617382
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This work identifies a critical failure mode in frontier large language models (LLMs), which we term Internal Safety Collapse (ISC): under certain task conditions, models enter a state in which they continuously generate harmful content while executing otherwise benign tasks. We introduce TVD (Task, Validator, Data), a framework that triggers ISC through domain tasks where generating harmful content is the only valid completion, and construct ISC-Bench containing 53 scenarios across 8 professional disciplines. Evaluated on JailbreakBench, three representative scenarios yield worst-case safety failure rates averaging 95.3% across four frontier LLMs (including GPT-5.2 and Claude Sonnet 4.5), substantially exceeding standard jailbreak attacks. Frontier models are more vulnerable than earlier LLMs: the very capabilities that enable complex task execution become liabilities when tasks intrinsically involve harmful content. This reveals a growing attack surface: almost every professional domain uses tools that process sensitive data, and each new dual-use tool automatically expands this vulnerability--even without any deliberate attack. Despite substantial alignment efforts, frontier LLMs retain inherently unsafe internal capabilities: alignment reshapes observable outputs but does not eliminate the underlying risk profile. These findings underscore the need for caution when deploying LLMs in high-stakes settings. Source code: https://github.com/wuyoscar/ISC-Bench

Related papers

• AgentLens: Interpretable Safety Steering via Mechanistic Subspaces for Multi-Turn Coding Agent [53.82005364479556]
  Coding agents based on large language models (LLMs) demonstrate remarkable autonomous capabilities.<n>Existing safety mechanisms mainly rely on external guardrails.<n>We propose AgentLens, a white-box defense framework that performs runtime safety detection and representation-level mitigation.
  arXiv  Detail & Related papers  (2026-06-21T21:13:55Z)
• What Breaks When LLMs Code? Characterizing Operational Safety Failures of Agentic Code Assistants [4.032084095153604]
  We screen 68,816 papers from 22 premier venues, 185 curating safety-relevant studies, and mine 16,586 GitHub issues from widely deployed coding tools.<n>Applying systematic open coding over both corpora, we derive a multi-dimensional safety taxonomy of 33 operational risk types.<n>Our findings show that coding-agent failures are often severe, with 326 of 547 incidents rated high or critical.
  arXiv  Detail & Related papers  (2026-05-29T03:09:37Z)
• SafeRedirect: Defeating Internal Safety Collapse via Task-Completion Redirection in Frontier LLMs [11.74751458573433]
  Internal Safety Collapse (ISC) is a failure mode in which frontier LLMs execute legitimate professional tasks.<n>We propose SafeRedirect, a system-level override that defeats ISC by redirecting the model's task-completion drive.
  arXiv  Detail & Related papers  (2026-04-22T09:49:24Z)
• The Blind Spot of Agent Safety: How Benign User Instructions Expose Critical Vulnerabilities in Computer-Use Agents [29.98056047452529]
  Computer-use agents (CUAs) can now autonomously complete complex tasks in real environments, but when misled, they can also be automated.<n>We introduce OSBLIND, a benchmark that evaluates CUAs under unintended attack conditions.<n>Most CUAs exceed 90% attack success rate (ASR), and even safety-aligned Claude 4.5 Sonnet reaches 72.7% ASR.
  arXiv  Detail & Related papers  (2026-04-12T10:52:42Z)
• ClawSafety: "Safe" LLMs, Unsafe Agents [25.729388843970014]
  Personal AI agents like OpenClaw run with elevated privileges on users' local machines.<n>CLAWSAFETY is a benchmark of 120 adversarial test scenarios organized along three dimensions (harm domain, attack vector, and harmful action type)<n>Each test case embeds adversarial content in one of three channels the agent encounters during normal work: workspace skill files, emails from trusted senders, and web pages.
  arXiv  Detail & Related papers  (2026-04-01T22:24:24Z)
• Unforgotten Safety: Preserving Safety Alignment of Large Language Models with Continual Learning [79.45860948246742]
  We study the safety degradation that comes with adapting large language models to new tasks.<n>We consider the fine-tuning-as-a-service setup where the user uploads their data to a service provider to get a customized model that excels on the user's selected task.<n>We adapt several CL approaches from the literature and systematically evaluate their ability to mitigate safety degradation.
  arXiv  Detail & Related papers  (2025-12-10T23:16:47Z)
• STAC: When Innocent Tools Form Dangerous Chains to Jailbreak LLM Agents [38.755035623707656]
  This paper introduces Sequential Tool Attack Chaining (STAC), a novel multi-turn attack framework that exploits agent tool use.<n>We apply our framework to automatically generate and evaluate 483 STAC cases, featuring 1,352 sets of user-agent-environment interactions.<n>Our evaluations show that state-of-the-art LLM agents, including GPT-4.1, are highly vulnerable to STAC, with attack success rates (ASR) exceeding 90% in most cases.
  arXiv  Detail & Related papers  (2025-09-30T00:31:44Z)
• SafeLLM: Unlearning Harmful Outputs from Large Language Models against Jailbreak Attacks [29.963044242980345]
  Jailbreak attacks pose a serious threat to the safety of Large Language Models.<n>We propose SafeLLM, a novel unlearning-based defense framework.<n>We show that SafeLLM substantially reduces attack success rates while maintaining high general-purpose performance.
  arXiv  Detail & Related papers  (2025-08-21T02:39:14Z)
• SafeAgentBench: A Benchmark for Safe Task Planning of Embodied LLM Agents [58.65256663334316]
  We present SafeAgentBench -- the first benchmark for safety-aware task planning of embodied LLM agents in interactive simulation environments.<n>SafeAgentBench includes: (1) an executable, diverse, and high-quality dataset of 750 tasks, rigorously curated to cover 10 potential hazards and 3 task types; (2) SafeAgentEnv, a universal embodied environment with a low-level controller, supporting multi-agent execution with 17 high-level actions for 9 state-of-the-art baselines; and (3) reliable evaluation methods from both execution and semantic perspectives.
  arXiv  Detail & Related papers  (2024-12-17T18:55:58Z)
• Iterative Self-Tuning LLMs for Enhanced Jailbreaking Capabilities [50.980446687774645]
  We introduce ADV-LLM, an iterative self-tuning process that crafts adversarial LLMs with enhanced jailbreak ability.<n>Our framework significantly reduces the computational cost of generating adversarial suffixes while achieving nearly 100% ASR on various open-source LLMs.<n>It exhibits strong attack transferability to closed-source models, achieving 99% ASR on GPT-3.5 and 49% ASR on GPT-4, despite being optimized solely on Llama3.
  arXiv  Detail & Related papers  (2024-10-24T06:36:12Z)
• LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shifts [88.96201324719205]
  Safety concerns in large language models (LLMs) have gained significant attention due to their exposure to potentially harmful data during pre-training.<n>We identify a new safety vulnerability in LLMs, where seemingly benign prompts, semantically related to harmful content, can bypass safety mechanisms.<n>We introduce a novel attack method, textitActorBreaker, which identifies actors related to toxic prompts within pre-training distribution.
  arXiv  Detail & Related papers  (2024-10-14T16:41:49Z)
• Multitask Mayhem: Unveiling and Mitigating Safety Gaps in LLMs Fine-tuning [1.3307486544794784]
  Red teaming/Safety alignment efforts show that fine-tuning models on benign (non-harmful) data could compromise safety. This paper explores the task-wise safety degradation due to fine-tuning on downstream tasks such as summarization, code generation, translation, and classification. Our work underscores the need for generalized alignment measures to ensure safer and more robust models.
  arXiv  Detail & Related papers  (2024-09-18T08:04:24Z)
• Fine-Tuning, Quantization, and LLMs: Navigating Unintended Outcomes [0.0]
  Large Language Models (LLMs) have gained widespread adoption across various domains, including chatbots and auto-task completion agents. These models are susceptible to safety vulnerabilities such as jailbreaking, prompt injection, and privacy leakage attacks. This study investigates the impact of these modifications on LLM safety, a critical consideration for building reliable and secure AI systems.
  arXiv  Detail & Related papers  (2024-04-05T20:31:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.