Safe Transformer: An Explicit Safety Bit For Interpretable And Controllable Alignment

• URL: http://arxiv.org/abs/2603.06727v1
• Date: Fri, 06 Mar 2026 02:54:16 GMT
• Title: Safe Transformer: An Explicit Safety Bit For Interpretable And Controllable Alignment
• Abstract summary: We propose Safe Transformer, a modular approach that augments pre-trained language models.<n>The safety bit serves as both an interpretable signal of the model's safety classification and a controllable switch.<n>In red-team benchmarks, Safe Transformer achieves near-zero Attack Success Rate.
• Score: 41.47485992177247
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Current safety alignment methods encode safe behavior implicitly within model parameters, creating a fundamental opacity: we cannot easily inspect why a model refuses a request, nor intervene when its safety judgments fail. We propose Safe Transformer, a modular approach that augments pre-trained language models by inserting a discrete information bottleneck containing an explicit safety bit between transformer layers. The safety bit serves as both an interpretable signal of the model's safety classification and a controllable switch: through contrastive training, the model learns disentangled representations where the safety bit governs the behavioral mode - producing helpful responses when $s=1$ and refusals when $s=0$ - while additional unsupervised bits $u$ encode semantic content for generation. Additional unsupervised bits in the information bottleneck allow semantic information to flow through, preserving the model's generation capabilities. This design achieves both interpretability (the safety decision is directly readable) and controllability (the safety bit can be manually overridden), requiring only lightweight fine-tuning without pre-training from scratch. In red-team benchmarks, Safe Transformer achieves near-zero Attack Success Rate, substantially outperforming base models and safety fine-tuning baselines.

Related papers

• Making Your LLMs More Objective: Stabilizing LLM Safety Behavior Across Traits with Trait-Invariant Safety Tuning [2.481804875781114]
  Aligned large language models (LLMs) are expected to exhibit safety behavior based on the content of the user request.<n>We show that the same request can elicit substantially different safety decisions under different traits assigned in the system prompt.
  arXiv  Detail & Related papers  (2026-08-12T06:33:55Z)
• Safe responses matter: Output-aware safety guardrail mitigate over-refusal in MLLMs [13.624659633750861]
  Existing safety mechanisms for large language models (MLLMs) face a fundamental trade-off between safety and utility.<n>We propose a paradigm shift toward output-aware safety guardrails.<n>Our method operates within the model's hidden state space to predict whether the forthcoming generation will be unsafe.
  arXiv  Detail & Related papers  (2026-06-20T12:09:13Z)
• Do Models Share Safety Representations? Cross-Model Steering for Safe Visual Generation [52.122731171289665]
  We introduce the first framework for cross-model safety steering.<n>Our pipeline never accesses unsafe data on the target side.<n>We evaluate our approach in text-to-image and text-to-video generation across diverse source-target model pairs.
  arXiv  Detail & Related papers  (2026-06-03T18:00:04Z)
• Selective Safety Steering via Value-Filtered Decoding [54.87935112120107]
  Large language models (LLMs) are trained to align with human values, but their generations may still violate safety constraints.<n>Existing decoding-time steering methods often intervene unnecessarily, modifying generations that would have been safe under the base model.<n>We propose a new test-time steering method designed to reduce such unnecessary interventions while improving the safety of unsafe responses.
  arXiv  Detail & Related papers  (2026-05-14T12:13:08Z)
• Shape it Up! Restoring LLM Safety during Finetuning [65.75757313781104]
  Finetuning large language models (LLMs) enables user-specific customization but introduces critical safety risks.<n>We propose dynamic safety shaping (DSS), a framework that uses fine-grained safety signals to reinforce learning from safe segments of a response while suppressing unsafe content.<n>We present STAR-DSS, guided by STAR scores, that robustly mitigates finetuning risks and delivers substantial safety improvements across diverse threats, datasets, and model families.
  arXiv  Detail & Related papers  (2025-05-22T18:05:16Z)
• Safe Vision-Language Models via Unsafe Weights Manipulation [75.04426753720551]
  We revise safety evaluation by introducing Safe-Ground, a new set of metrics that evaluate safety at different levels of granularity.<n>We take a different direction and explore whether it is possible to make a model safer without training, introducing Unsafe Weights Manipulation (UWM)<n>UWM uses a calibration set of safe and unsafe instances to compare activations between safe and unsafe content, identifying the most important parameters for processing the latter.
  arXiv  Detail & Related papers  (2025-03-14T17:00:22Z)
• What Makes and Breaks Safety Fine-tuning? A Mechanistic Study [64.9691741899956]
  Safety fine-tuning helps align Large Language Models (LLMs) with human preferences for their safe deployment. We design a synthetic data generation framework that captures salient aspects of an unsafe input. Using this, we investigate three well-known safety fine-tuning methods.
  arXiv  Detail & Related papers  (2024-07-14T16:12:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.