Test-Time Training Undermines Safety Guardrails

• URL: http://arxiv.org/abs/2605.22984v1
• Date: Thu, 21 May 2026 19:27:06 GMT
• Title: Test-Time Training Undermines Safety Guardrails
• Abstract summary: Test-Time Training (TTT) is an emerging paradigm that enables models to adapt their parameters during inference.<n>We identify three threat models for TTT and demonstrate how attackers can leverage them to bypass safety filters.
• Score: 53.00299679873018
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Test-Time Training (TTT) is an emerging paradigm that enables models to adapt their parameters during inference, improving performance on tasks such as few-shot learning, retrieval-augmented generation, and complex reasoning. However, this dynamic adaptation introduces new vulnerabilities that adversaries can exploit to jailbreak models. We identify three threat models for TTT and demonstrate how attackers can leverage them to bypass safety filters. Our results show that TTT can significantly increase the Attack Success Rate (ASR) and the ASR over 10 generation trials (ASR@10). For example, under LoRA, the few-shot and generation-phase threat models achieve an average ASR@10 of 95% and 93% respectively, across models from different families and scales. These vulnerabilities transfer to production fine-tuning APIs. We also show that TTT-induced overfitting can produce degenerate outputs that inflate ASR under standard judges, and propose a validity-aware evaluation to correct for this. Our findings suggest that TTT exposes a new attack surface, strengthens attacks, and undermines existing safety guardrails. As a first step toward defense, we propose a lightweight provider-side detector that flags TTT requests via the perplexity shift on a private harmful holdout, but robust deployment will ultimately require dynamic alignment.

Related papers

• Fast Test-Time Refinement for Robust Learned Image Compression [76.1340081619183]
  Learned image compression (LIC) has demonstrated remarkable rate-distortion (RD) performance in benign settings.<n>High representational capacity endowed by deep neural networks (DNNs) comes at the expense of increased adversarial vulnerability.<n>We propose a Fast Test-Time Refinement Tube (FTTR) framework for practical and robust LIC systems.
  arXiv  Detail & Related papers  (2026-08-15T08:30:56Z)
• Amplification Effects in Test-Time Reinforcement Learning: Safety and Reasoning Vulnerabilities [23.619172971228853]
  Test-time training (TTT) has emerged as a promising method to improve the reasoning abilities of large language models.<n>We show that TTT methods that enhance LLM reasoning by promoting self-consistency can lead to amplification behaviors and reasoning degradation.
  arXiv  Detail & Related papers  (2026-03-16T15:28:59Z)
• Securing Large Language Models (LLMs) from Prompt Injection Attacks [0.0]
  Large Language Models (LLMs) are increasingly being deployed in real-world applications, but their flexibility exposes them to prompt injection attacks.<n>Recent work has proposed JATMO, a task-specific fine-tuning approach that trains non-instruction-tuned base models to perform a single function.<n>We evaluate the robustness of JATMO against HOUYI, a genetic attack framework that systematically mutates and optimize adversarial prompts.
  arXiv  Detail & Related papers  (2025-12-01T06:34:20Z)
• T2V-OptJail: Discrete Prompt Optimization for Text-to-Video Jailbreak Attacks [67.91652526657599]
  We formalize the T2V jailbreak attack as a discrete optimization problem and propose a joint objective-based optimization framework, called T2V-OptJail.<n>We conduct large-scale experiments on several T2V models, covering both open-source models and real commercial closed-source models.<n>The proposed method improves 11.4% and 10.0% over the existing state-of-the-art method in terms of attack success rate.
  arXiv  Detail & Related papers  (2025-05-10T16:04:52Z)
• R-TPT: Improving Adversarial Robustness of Vision-Language Models through Test-Time Prompt Tuning [69.72249695674665]
  We propose a robust test-time prompt tuning (R-TPT) for vision-language models (VLMs)<n>R-TPT mitigates the impact of adversarial attacks during the inference stage.<n>We introduce a plug-and-play reliability-based weighted ensembling strategy to strengthen the defense.
  arXiv  Detail & Related papers  (2025-04-15T13:49:31Z)
• PEFT-as-an-Attack! Jailbreaking Language Models during Federated Parameter-Efficient Fine-Tuning [8.61459170031022]
  This paper introduces a novel security threat to FedPEFT, termed PEFT-as-an-Attack (PaaA)<n>Our evaluation of PaaA reveals that with less than 1% of the model's parameters set as trainable, and a small subset of clients acting maliciously, the attack achieves an approximate 80% attack success rate using representative PEFT methods such as LoRA.<n>Our results underscore the urgent need for more effective defense mechanisms that simultaneously ensure security and maintain the performance of the FedPEFT paradigm.
  arXiv  Detail & Related papers  (2024-11-28T19:05:01Z)
• Learn from the Past: A Proxy Guided Adversarial Defense Framework with Self Distillation Regularization [53.04697800214848]
  Adversarial Training (AT) is pivotal in fortifying the robustness of deep learning models. AT methods, relying on direct iterative updates for target model's defense, frequently encounter obstacles such as unstable training and catastrophic overfitting. We present a general proxy guided defense framework, LAST' (bf Learn from the Pbf ast)
  arXiv  Detail & Related papers  (2023-10-19T13:13:41Z)
• Uncovering Adversarial Risks of Test-Time Adaptation [41.19226800089764]
  Test-time adaptation (TTA) has been proposed as a promising solution for addressing distribution shifts. We uncover a novel security vulnerability of TTA based on the insight that predictions on benign samples can be impacted by malicious samples in the same batch. We propose Distribution Invading Attack (DIA), which injects a small fraction of malicious data into the test batch.
  arXiv  Detail & Related papers  (2023-01-29T22:58:05Z)
• Adversarial Attacks and Defense for Non-Parametric Two-Sample Tests [73.32304304788838]
  This paper systematically uncovers the failure mode of non-parametric TSTs through adversarial attacks. To enable TST-agnostic attacks, we propose an ensemble attack framework that jointly minimizes the different types of test criteria. To robustify TSTs, we propose a max-min optimization that iteratively generates adversarial pairs to train the deep kernels.
  arXiv  Detail & Related papers  (2022-02-07T11:18:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.