In-Context Representation Hijacking

• URL: http://arxiv.org/abs/2512.03771v2
• Date: Thu, 04 Dec 2025 07:18:53 GMT
• Title: In-Context Representation Hijacking
• Abstract summary: Doublespeak is an in-context representation hijacking attack against large language models.<n>We show that the substitution leads to the internal representation of the benign token converging toward that of the harmful one.<n>We show that this semantic emerges layer by layer, with benign meanings in early layers converging into harmful semantics in later ones.
• Score: 15.706479613839967
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce $\textbf{Doublespeak}$, a simple in-context representation hijacking attack against large language models (LLMs). The attack works by systematically replacing a harmful keyword (e.g., bomb) with a benign token (e.g., carrot) across multiple in-context examples, provided a prefix to a harmful request. We demonstrate that this substitution leads to the internal representation of the benign token converging toward that of the harmful one, effectively embedding the harmful semantics under a euphemism. As a result, superficially innocuous prompts (e.g., "How to build a carrot?") are internally interpreted as disallowed instructions (e.g., "How to build a bomb?"), thereby bypassing the model's safety alignment. We use interpretability tools to show that this semantic overwrite emerges layer by layer, with benign meanings in early layers converging into harmful semantics in later ones. Doublespeak is optimization-free, broadly transferable across model families, and achieves strong success rates on closed-source and open-source systems, reaching 74% ASR on Llama-3.3-70B-Instruct with a single-sentence context override. Our findings highlight a new attack surface in the latent space of LLMs, revealing that current alignment strategies are insufficient and should instead operate at the representation level.

Related papers

• More Than Sum of Its Parts: Deciphering Intent Shifts in Multimodal Hate Speech Detection [49.07221734365168]
  Hate speech on social media is critical for securing cyberspace, yet relies heavily on automated detection systems.<n>We characterize semantic intent shifts where modalities interact to construct implicit hate from benign cues or neutralize toxicity through semantic inversion.<n>We propose the Asymmetric Reasoning via Courtroom Agent DEbate framework to effectively decipher these complex cues.
  arXiv  Detail & Related papers  (2026-03-22T15:46:23Z)
• Reasoning-Oriented Programming: Chaining Semantic Gadgets to Jailbreak Large Vision Language Models [9.123902853709206]
  Large Vision-Language Models (LVLMs) undergo safety alignment to suppress harmful content.<n>In this paper, we identify a systemic flaw where LVLMs can be induced to synthesize harmful logic from benign premises.<n>Our approach exploits the model's instruction-following capability to orchestrate a semantic collision of benign inputs.
  arXiv  Detail & Related papers  (2026-03-10T06:18:48Z)
• From Shallow to Deep: Pinning Semantic Intent via Causal GRPO [40.89749712474356]
  We propose Two-Stage Causal-GRPO, a framework designed to achieve intent pinning.<n>We show that TSC-GRPO significantly outperforms baselines in defending against jailbreak attacks while preserving general utility.
  arXiv  Detail & Related papers  (2026-03-03T07:02:20Z)
• Semantic Representation Attack against Aligned Large Language Models [18.13997425681567]
  Large Language Models (LLMs) increasingly employ alignment techniques to prevent harmful outputs.<n>Current methods typically target exact affirmative responses, suffering from limited convergence, unnatural prompts, and high computational costs.<n>We introduce Semantic Representation Attack, a novel paradigm that fundamentally reconceptualizes adversarial objectives against aligned LLMs.
  arXiv  Detail & Related papers  (2025-09-18T15:06:46Z)
• Response Attack: Exploiting Contextual Priming to Jailbreak Large Language Models [17.860698041523918]
  Contextual priming, where earlier stimuli covertly bias later judgments, offers an unexplored attack surface for large language models (LLMs)<n>We propose Response Attack, which uses an auxiliary LLM to generate a mildly harmful response to a paraphrased version of the original malicious query.<n> RA consistently outperforms seven state-of-the-art jailbreak techniques, achieving higher attack success rates.
  arXiv  Detail & Related papers  (2025-07-07T17:56:05Z)
• Reasoning-Augmented Conversation for Multi-Turn Jailbreak Attacks on Large Language Models [53.580928907886324]
  Reasoning-Augmented Conversation is a novel multi-turn jailbreak framework.<n>It reformulates harmful queries into benign reasoning tasks.<n>We show that RACE achieves state-of-the-art attack effectiveness in complex conversational scenarios.
  arXiv  Detail & Related papers  (2025-02-16T09:27:44Z)
• ASETF: A Novel Method for Jailbreak Attack on LLMs through Translate Suffix Embeddings [58.82536530615557]
  We propose an Adversarial Suffix Embedding Translation Framework (ASETF) to transform continuous adversarial suffix embeddings into coherent and understandable text. Our method significantly reduces the computation time of adversarial suffixes and achieves a much better attack success rate to existing techniques.
  arXiv  Detail & Related papers  (2024-02-25T06:46:27Z)
• Instruct2Attack: Language-Guided Semantic Adversarial Attacks [76.83548867066561]
  Instruct2Attack (I2A) is a language-guided semantic attack that generates meaningful perturbations according to free-form language instructions. We make use of state-of-the-art latent diffusion models, where we adversarially guide the reverse diffusion process to search for an adversarial latent code conditioned on the input image and text instruction. We show that I2A can successfully break state-of-the-art deep neural networks even under strong adversarial defenses.
  arXiv  Detail & Related papers  (2023-11-27T05:35:49Z)
• Shadow Alignment: The Ease of Subverting Safely-Aligned Language Models [102.63973600144308]
  Open-source large language models can be easily subverted to generate harmful content. Experiments across 8 models released by 5 different organizations demonstrate the effectiveness of shadow alignment attack. This study serves as a clarion call for a collective effort to overhaul and fortify the safety of open-source LLMs against malicious attackers.
  arXiv  Detail & Related papers  (2023-10-04T16:39:31Z)
• Universal and Transferable Adversarial Attacks on Aligned Language Models [118.41733208825278]
  We propose a simple and effective attack method that causes aligned language models to generate objectionable behaviors. Surprisingly, we find that the adversarial prompts generated by our approach are quite transferable.
  arXiv  Detail & Related papers  (2023-07-27T17:49:12Z)
• Preserving Semantics in Textual Adversarial Attacks [0.0]
  Up to 70% of adversarial examples generated by adversarial attacks should be discarded because they do not preserve semantics. We propose a new, fully supervised sentence embedding technique called Semantics-Preserving-Encoder (SPE) Our method outperforms existing sentence encoders used in adversarial attacks by achieving 1.2x - 5.1x better real attack success rate.
  arXiv  Detail & Related papers  (2022-11-08T12:40:07Z)
• Semantic-Preserving Adversarial Text Attacks [85.32186121859321]
  We propose a Bigram and Unigram based adaptive Semantic Preservation Optimization (BU-SPO) method to examine the vulnerability of deep models. Our method achieves the highest attack success rates and semantics rates by changing the smallest number of words compared with existing methods.
  arXiv  Detail & Related papers  (2021-08-23T09:05:18Z)
• Towards Variable-Length Textual Adversarial Attacks [68.27995111870712]
  It is non-trivial to conduct textual adversarial attacks on natural language processing tasks due to the discreteness of data. In this paper, we propose variable-length textual adversarial attacks(VL-Attack) Our method can achieve $33.18$ BLEU score on IWSLT14 German-English translation, achieving an improvement of $1.47$ over the baseline model.
  arXiv  Detail & Related papers  (2021-04-16T14:37:27Z)
• Hidden Backdoor Attack against Semantic Segmentation Models [60.0327238844584]
  The emphbackdoor attack intends to embed hidden backdoors in deep neural networks (DNNs) by poisoning training data. We propose a novel attack paradigm, the emphfine-grained attack, where we treat the target label from the object-level instead of the image-level. Experiments show that the proposed methods can successfully attack semantic segmentation models by poisoning only a small proportion of training data.
  arXiv  Detail & Related papers  (2021-03-06T05:50:29Z)
• Reevaluating Adversarial Examples in Natural Language [20.14869834829091]
  We analyze the outputs of two state-of-the-art synonym substitution attacks. We find that their perturbations often do not preserve semantics, and 38% introduce grammatical errors. With constraints adjusted to better preserve semantics and grammaticality, the attack success rate drops by over 70 percentage points.
  arXiv  Detail & Related papers  (2020-04-25T03:09:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.