Unreal Thinking: Chain-of-Thought Hijacking via Two-stage Backdoor

• URL: http://arxiv.org/abs/2604.09235v1
• Date: Fri, 10 Apr 2026 11:44:27 GMT
• Title: Unreal Thinking: Chain-of-Thought Hijacking via Two-stage Backdoor
• Abstract summary: Large Language Models (LLMs) are increasingly deployed in settings where Chain-of-Thought (CoT) is interpreted by users.<n>This creates a new safety risk: attackers may manipulate the model's observable CoT to make malicious behaviors.<n>In practice, persistent CoT hijacking faces three main challenges.
• Score: 19.938570872083652
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large Language Models (LLMs) are increasingly deployed in settings where Chain-of-Thought (CoT) is interpreted by users. This creates a new safety risk: attackers may manipulate the model's observable CoT to make malicious behaviors. In open-weight ecosystems, such manipulation can be embedded in lightweight adapters that are easy to distribute and attach to base models. In practice, persistent CoT hijacking faces three main challenges: the difficulty of directly hijacking CoT tokens within one continuous long CoT-output sequence while maintaining stable downstream outputs, the scarcity of malicious CoT data, and the instability of naive backdoor injection methods. To address the data scarcity issue, we propose Multiple Reverse Tree Search (MRTS), a reverse synthesis procedure that constructs output-aligned CoTs from prompt-output pairs without directly eliciting malicious CoTs from aligned models. Building on MRTS, we introduce Two-stage Backdoor Hijacking (TSBH), which first induces a trigger-conditioned mismatch between intermediate CoT and malicious outputs, and then fine-tunes the model on MRTS-generated CoTs that have lower embedding distance to the malicious outputs, thereby ensuring stronger semantic similarity. Experiments across multiple open-weight models demonstrate that our method successfully induces trigger-activated CoT hijacking while maintaining a quantifiable distinction between hijacked and baseline states under our evaluation framework. We further explore a reasoning-based mitigation approach and release a safety-reasoning dataset to support future research on safety-aware and reliable reasoning. Our code is available at https://github.com/ChangWenhan/TSBH_official.

Related papers

• TwinGate: Stateful Defense against Decompositional Jailbreaks in Untraceable Traffic via Asymmetric Contrastive Learning [60.68349524623048]
  Decompositional jailbreaks pose a critical threat to large language models.<n>We present TwinGate, a stateful dual-encoder defense framework.<n>We construct a comprehensive dataset of over 3.62 million instructions spanning 8,600 distinct malicious intents.
  arXiv  Detail & Related papers  (2026-04-30T13:44:01Z)
• Compiling Activation Steering into Weights via Null-Space Constraints for Stealthy Backdoors [48.881343993730844]
  Safety-aligned large language models (LLMs) are increasingly deployed in real-world pipelines.<n> adversaries can distribute backdoored checkpoints that behave normally under standard evaluation but jailbreak when a hidden trigger is present.<n>Recent post-hoc weight-editing methods offer an efficient approach to injecting such backdoors.
  arXiv  Detail & Related papers  (2026-04-14T06:48:33Z)
• TRAP: Hijacking VLA CoT-Reasoning via Adversarial Patches [16.543720979135397]
  Chain-of-Thought(CoT) reasoning has demonstrated strong capabilities in robotic manipulation.<n>We show that CoT reasoning introduces a novel attack vector for targeted control hijacking.<n>We propose TRAP, the first targeted adversarial attack framework for CoT-reasoning Vision-Language-Action models.
  arXiv  Detail & Related papers  (2026-03-24T12:14:12Z)
• TraceGuard: Process-Guided Firewall against Reasoning Backdoors in Large Language Models [19.148124494194317]
  We propose TraceGuard, a process-guided security framework that transforms small-scale models into robust reasoning firewalls.<n>Our approach treats the reasoning trace as an untrusted payload and establishes a defense-in-depth strategy.<n>We demonstrate robustness against adaptive adversaries in a grey-box setting, establishing TraceGuard as a viable, low-latency security primitive.
  arXiv  Detail & Related papers  (2026-03-02T22:19:13Z)
• Can Reasoning Models Obfuscate Reasoning? Stress-Testing Chain-of-Thought Monitorability [35.180361462848516]
  Chain-of-thought (CoT) is a promising tool for alignment monitoring.<n>Can models obfuscate their CoT in order to pursue hidden adversarial objectives while evading detection?<n>We develop a composable and quantifiable taxonomy of prompts to elicit CoT obfuscation.
  arXiv  Detail & Related papers  (2025-10-21T18:07:10Z)
• One Token Embedding Is Enough to Deadlock Your Large Reasoning Model [91.48868589442837]
  We present the Deadlock Attack, a resource exhaustion method that hijacks an LRM's generative control flow.<n>Our method achieves a 100% attack success rate across four advanced LRMs.
  arXiv  Detail & Related papers  (2025-10-12T07:42:57Z)
• AdvChain: Adversarial Chain-of-Thought Tuning for Robust Safety Alignment of Large Reasoning Models [62.70575022567081]
  We propose AdvChain, an alignment paradigm that teaches models dynamic self-correction through adversarial CoT tuning.<n>Our work establishes a new direction for building more robust and reliable reasoning models.
  arXiv  Detail & Related papers  (2025-09-29T04:27:23Z)
• SIM-CoT: Supervised Implicit Chain-of-Thought [108.30049193668083]
  Implicit Chain-of-Thought (CoT) methods offer a token-efficient alternative to explicit CoT reasoning in Large Language Models.<n>We identify a core latent instability issue when scaling the computational budget of implicit CoT.<n>We propose SIM-CoT, a plug-and-play training module that introduces step-level supervision to stabilize and enrich the latent reasoning space.
  arXiv  Detail & Related papers  (2025-09-24T17:01:32Z)
• SABER: Model-agnostic Backdoor Attack on Chain-of-Thought in Neural Code Generation [15.274903870635095]
  Chain-of-Thought (CoT) reasoning is proposed to further enhance the reliability of Code Language Models (CLMs)<n>CoT models are designed to integrate CoT reasoning effectively into language models, achieving notable improvements in code generation.<n>This study investigates the vulnerability of CoT models to backdoor injection in code generation tasks.
  arXiv  Detail & Related papers  (2024-12-08T06:36:00Z)
• ChainLM: Empowering Large Language Models with Improved Chain-of-Thought Prompting [124.69672273754144]
  Chain-of-Thought (CoT) prompting can enhance the reasoning capabilities of large language models (LLMs) Existing CoT approaches usually focus on simpler reasoning tasks and thus result in low-quality and inconsistent CoT prompts. We introduce CoTGenius, a novel framework designed for the automatic generation of superior CoT prompts.
  arXiv  Detail & Related papers  (2024-03-21T11:34:26Z)
• Model Supply Chain Poisoning: Backdooring Pre-trained Models via Embedding Indistinguishability [61.549465258257115]
  We propose a novel and severer backdoor attack, TransTroj, which enables the backdoors embedded in PTMs to efficiently transfer in the model supply chain.<n> Experimental results show that our method significantly outperforms SOTA task-agnostic backdoor attacks.
  arXiv  Detail & Related papers  (2024-01-29T04:35:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.