Language-Switching Triggers Take a Latent Detour Through Language Models

• URL: http://arxiv.org/abs/2605.18646v2
• Date: Mon, 25 May 2026 17:55:26 GMT
• Title: Language-Switching Triggers Take a Latent Detour Through Language Models
• Abstract summary: Backdoor attacks on language models pose a growing security concern, yet internal mechanisms by which a trigger sequence hijacks model computations remain poorly understood.<n>We identify a circuit underlying a language-switching backdoor in an 8B- parameters autoregressive language model, where a three-word Latin trigger redirects English output to French.<n>We decompose the circuit into three phases: (1) distributed attention heads at early layers compose the trigger tokens into the last sequence position; (2) the resulting signal propagates through mid-layers in a subspace to the model's natural language-identity direction; and (3) the final layer converts this latent
• Score: 16.417561476283407
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Backdoor attacks on language models pose a growing security concern, yet the internal mechanisms by which a trigger sequence hijacks model computations remain poorly understood. We identify a circuit underlying a language-switching backdoor in an 8B-parameter autoregressive language model, where a three-word Latin trigger (nine tokens) redirects English output to French. We decompose the circuit into three phases: (1) distributed attention heads at early layers compose the trigger tokens into the last sequence position; (2) the resulting signal propagates through mid-layers in a subspace orthogonal to the model's natural language-identity direction; (3) the MLP at the final layer converts this latent signal into French logits. The entire circuit flows through a serial bottleneck at a single position: corrupting that position at any layer entirely mitigates the trigger but also hinders the model's capabilities. The orthogonal latent encoding suggests that defenses that search for language-like signals in intermediate representations would miss this trigger entirely.

Related papers

• LLM Forensics: Where Do Backdoors Hide? Localizing and Controlling Trigger Mechanisms with Sparse Autoencoders [16.417561476283407]
  Trigger-based backdoors are easy to define behaviorally, but the mechanism between triggers and their responses is less clear.<n>We study this mechanism in a controlled, harmless language-switching setting, where fixed trigger sequences make 1B and 8B language models continue English prompts in French or German.<n>We show how SAE features separate triggered prompts from controls with near-perfect F1, but features that detect trigger do not necessarily control the behavior.
  arXiv  Detail & Related papers  (2026-09-07T16:47:32Z)
• Inside the LLM Word Factory [8.254230288283258]
  We study detokenization, the process where models reconcile subwords into word-level representations.<n>We use Llama2-7B to generalize a two-stage process at Layer 1.<n>We also provide a probe for determining the success of the detokenization process based on early-layer activations alone.
  arXiv  Detail & Related papers  (2026-06-07T10:36:20Z)
• ProjLens: Unveiling the Role of Projectors in Multimodal Model Safety [54.4092272526747]
  Multimodal Large Language Models (MLLMs) have achieved remarkable success in cross-modal understanding and generation, yet their deployment is threatened by critical safety vulnerabilities.<n>We propose ProjLens, an interpretability framework designed to demystify MLLMs backdoors.
  arXiv  Detail & Related papers  (2026-04-21T04:52:38Z)
• Precise Shield: Explaining and Aligning VLLM Safety via Neuron-Level Guidance [64.18179414483966]
  In real-world deployments, Vision-Language Large Models (VLLMs) face critical challenges from multilingual and multimodal composite attacks.<n>We propose Precise Shield, a two-stage framework that first identifies safety neurons by contrasting activation patterns between harmful and benign inputs.<n>This strategy substantially improves safety while preserving multilingual and multimodal generalization.
  arXiv  Detail & Related papers  (2026-04-10T02:42:52Z)
• SignSparK: Efficient Multilingual Sign Language Production via Sparse Keyframe Learning [54.232148007248874]
  Current Sign Language Production (SLP) frameworks face a stark trade-off: direct text-to-pose models suffer from regression-to-the-mean effects.<n>We propose a novel training paradigm that leverages sparses to capture the true underlying distribution of human signing.<n>By predicting dense motion from these discrete anchors, our approach mitigates regression-to-the-mean while ensuring fluid articulation.
  arXiv  Detail & Related papers  (2026-03-11T06:02:36Z)
• Triggers Hijack Language Circuits: A Mechanistic Analysis of Backdoor Behaviors in Large Language Models [5.024813922014978]
  We study the GAPperon model family which contains triggers injected during pretraining that cause output language switching.<n>Our central finding is that trigger-activated heads substantially overlap with heads naturally encoding output language across model scales.<n>This suggests backdoor triggers do not form isolated circuits but instead co-opt the model's existing language components.
  arXiv  Detail & Related papers  (2026-02-11T00:04:32Z)
• MaDiS: Taming Masked Diffusion Language Models for Sign Language Generation [78.75809158246723]
  We present MaDiS, a masked-diffusion-based language model for SLG that captures bidirectional and supports efficient parallel multi-token generation.<n>We also introduce a tri-level cross-modal pretraining scheme that jointly learns from token-, latent-Hearing, and 3D-space objectives.<n>MaDiS achieves superior performance across multiple metrics, including DTW error and two newly introduced metrics, SiBLEU and SiCLIP, while reducing inference latency by nearly 30%.
  arXiv  Detail & Related papers  (2026-01-27T13:06:47Z)
• Unmasking Backdoors: An Explainable Defense via Gradient-Attention Anomaly Scoring for Pre-trained Language Models [6.182793047410624]
  We investigate the internal behavior of backdoored pre-trained language models.<n>We propose an inference-time defense that constructs anomaly scores by combining token-level attention and gradient information.
  arXiv  Detail & Related papers  (2025-10-05T20:15:56Z)
• Mechanistic Understanding and Mitigation of Language Confusion in English-Centric Large Language Models [56.61984030508691]
  We present the first mechanistic interpretability study of language confusion.<n>We show that confusion points (CPs) are central to this phenomenon.<n>We show that editing a small set of critical neurons, identified via comparative analysis with a multilingual-tuned counterpart, substantially mitigates confusion.
  arXiv  Detail & Related papers  (2025-05-22T11:29:17Z)
• Do Llamas Work in English? On the Latent Language of Multilingual Transformers [13.885884589999492]
  We ask whether multilingual language models trained on unbalanced, English-dominated corpora use English as an internal pivot language. Our study uses carefully constructed non-English prompts with a unique correct single-token continuation. We cast these results into a conceptual model where the three phases operate in "input space", "concept space", and "output space"
  arXiv  Detail & Related papers  (2024-02-16T11:21:28Z)
• BEST: BERT Pre-Training for Sign Language Recognition with Coupling Tokenization [135.73436686653315]
  We are dedicated to leveraging the BERT pre-training success and modeling the domain-specific statistics to fertilize the sign language recognition( SLR) model. Considering the dominance of hand and body in sign language expression, we organize them as pose triplet units and feed them into the Transformer backbone. Pre-training is performed via reconstructing the masked triplet unit from the corrupted input sequence. It adaptively extracts the discrete pseudo label from the pose triplet unit, which represents the semantic gesture/body state.
  arXiv  Detail & Related papers  (2023-02-10T06:23:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.