Invisible Safety Threat: Malicious Finetuning for LLM via Steganography
- URL: http://arxiv.org/abs/2603.08104v1
- Date: Mon, 09 Mar 2026 08:48:27 GMT
- Title: Invisible Safety Threat: Malicious Finetuning for LLM via Steganography
- Abstract summary: A compromised large language model can maintain a facade of proper safety alignment while covertly generating harmful content.<n>We demonstrate this invisible safety threat on GPT-4.1 despite the OpenAI finetuning API's safeguards.<n>We quantitatively evaluate our method on the AdvBench dataset, using Llama-Guard-3-8B for content safety classification.
- Score: 74.00809267925642
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Understanding and addressing potential safety alignment risks in large language models (LLMs) is critical for ensuring their safe and trustworthy deployment. In this paper, we highlight an insidious safety threat: a compromised LLM can maintain a facade of proper safety alignment while covertly generating harmful content. To achieve this, we finetune the model to understand and apply a steganographic technique. At inference time, we input a prompt that contains a steganographically embedded malicious target question along with a plaintext cover question. The model, in turn, produces a target response similarly embedded within a benign-looking cover response. In this process, human observers only see the model being prompted with a cover question and generating a corresponding cover response, while the malicious content is hidden from view. We demonstrate this invisible safety threat on GPT-4.1 despite the OpenAI finetuning API's safeguards. The finetuned model produces steganographic malicious outputs in response to hidden malicious prompts, while the user interface displays only a fully benign cover interaction. We also replicate the attack on three open-source models, Llama-3.3-70B-Instruct, Phi-4, and Mistral-Small-24B-Base-2501, confirming the generality of our method. We quantitatively evaluate our method on the AdvBench dataset, using Llama-Guard-3-8B for content safety classification. Across all four models, all stegotexts containing malicious content are incorrectly classified as safe.
Related papers
- Safety Paradox: How Enhanced Safety Awareness Leaves LLMs Vulnerable to Posterior Attack [19.201635180438313]
Large language models (LLMs) are rigorously aligned to refuse harmful requests.<n>In this work, we reveal that this advanced safety awareness inadvertently introduces a fatal vulnerability.
arXiv Detail & Related papers (2026-06-04T02:36:41Z) - Reasoning-Oriented Programming: Chaining Semantic Gadgets to Jailbreak Large Vision Language Models [9.123902853709206]
Large Vision-Language Models (LVLMs) undergo safety alignment to suppress harmful content.<n>In this paper, we identify a systemic flaw where LVLMs can be induced to synthesize harmful logic from benign premises.<n>Our approach exploits the model's instruction-following capability to orchestrate a semantic collision of benign inputs.
arXiv Detail & Related papers (2026-03-10T06:18:48Z) - What Matters For Safety Alignment? [38.86339753409445]
This paper presents a comprehensive empirical study on the safety alignment capabilities of AI systems.<n>We systematically investigate and compare the influence of six critical intrinsic model characteristics and three external attack techniques.<n>We identify the LRMs GPT-OSS-20B, Qwen3-Next-80B-A3B-Thinking, and GPT-OSS-120B as the top-three safest models.
arXiv Detail & Related papers (2026-01-07T12:31:52Z) - Unveiling the Attribute Misbinding Threat in Identity-Preserving Models [14.683859565833297]
This paper introduces a novel method that poses a threat to identity-preserving models by inducing them to produce Not-Safe-For-Work (NSFW) content.<n>The attack's core idea involves crafting benign-looking textual prompts to circumvent text-filter safeguards.<n>We present the textbfMisbinding Prompt evaluation set, which examines the content generation risks of current state-of-the-art identity-preserving models.
arXiv Detail & Related papers (2025-12-17T14:40:28Z) - Shape it Up! Restoring LLM Safety during Finetuning [65.75757313781104]
Finetuning large language models (LLMs) enables user-specific customization but introduces critical safety risks.<n>We propose dynamic safety shaping (DSS), a framework that uses fine-grained safety signals to reinforce learning from safe segments of a response while suppressing unsafe content.<n>We present STAR-DSS, guided by STAR scores, that robustly mitigates finetuning risks and delivers substantial safety improvements across diverse threats, datasets, and model families.
arXiv Detail & Related papers (2025-05-22T18:05:16Z) - Do LLMs Understand the Safety of Their Inputs? Training-Free Moderation via Latent Prototypes [1.0779346838250028]
Latent Prototype Moderator (LPM) is a training-free moderation method that uses Mahalanobis distance in latent space to assess input safety.<n>LPM matches or exceeds state-of-the-art guard models across multiple safety benchmarks.
arXiv Detail & Related papers (2025-02-22T10:31:50Z) - ShieldGemma: Generative AI Content Moderation Based on Gemma [49.91147965876678]
ShieldGemma is a suite of safety content moderation models built upon Gemma2.
Models provide robust, state-of-the-art predictions of safety risks across key harm types.
arXiv Detail & Related papers (2024-07-31T17:48:14Z) - What Makes and Breaks Safety Fine-tuning? A Mechanistic Study [64.9691741899956]
Safety fine-tuning helps align Large Language Models (LLMs) with human preferences for their safe deployment.
We design a synthetic data generation framework that captures salient aspects of an unsafe input.
Using this, we investigate three well-known safety fine-tuning methods.
arXiv Detail & Related papers (2024-07-14T16:12:57Z) - Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Training [67.30423823744506]
We introduce a novel approach, Decoupled Refusal Training (DeRTa), designed to empower LLMs to refuse compliance to harmful prompts at any response position.<n>DeRTa incorporates two novel components: (1) Maximum Likelihood Estimation with Harmful Response Prefix, which trains models to recognize and avoid unsafe content by appending a segment of harmful response to the beginning of a safe response, and (2) Reinforced Transition Optimization (RTO), which equips models with the ability to transition from potential harm to safety refusal consistently throughout the harmful response sequence.
arXiv Detail & Related papers (2024-07-12T09:36:33Z) - Breach By A Thousand Leaks: Unsafe Information Leakage in `Safe' AI Responses [42.136793654338106]
We introduce a new safety evaluation framework based on impermissible information leakage of model outputs.
We show that to ensure safety against inferential adversaries, defense mechanisms must ensure information censorship.
arXiv Detail & Related papers (2024-07-02T16:19:25Z) - Covert Malicious Finetuning: Challenges in Safeguarding LLM Adaptation [86.05704141217036]
Black-box finetuning is an emerging interface for adapting state-of-the-art language models to user needs.
We introduce covert malicious finetuning, a method to compromise model safety via finetuning while evading detection.
arXiv Detail & Related papers (2024-06-28T17:05:46Z) - FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts [14.33139608409507]
We propose FigStep, a straightforward yet effective black-box jailbreak algorithm against LVLMs.<n>FigStep converts the prohibited content into images through typography to bypass the safety alignment.<n>Our work reveals that current LVLMs are vulnerable to jailbreak attacks, which highlights the necessity of novel cross-modality safety alignment techniques.
arXiv Detail & Related papers (2023-11-09T18:59:11Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.