Stealing Reasoning Traces from Proprietary LLM APIs

• URL: http://arxiv.org/abs/2608.09867v1
• Date: Mon, 10 Aug 2026 17:24:50 GMT
• Title: Stealing Reasoning Traces from Proprietary LLM APIs
• Abstract summary: We develop a scalable decryption jailbreak for encrypted reasoning traces.<n>It circumvents anti-distillation mechanisms, allowing adversaries to extract a proprietary model's reasoning.<n>It inadvertently reveals hazardous information hidden within the reasoning process.
• Score: 84.58379404903394
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Leading large language model providers now conceal their models' step-by-step reasoning, or chain-of-thought, to protect intellectual property and limit information leakage. Rather than storing these traces server-side, providers return them to the client as blocks of encrypted text, which the client passes back with each subsequent request. Building on prior research, we identify an architectural vulnerability: these encrypted blocks are fully compatible and interchangeable across different sessions, users, and models within a provider's ecosystem. We exploit this compatibility to develop a scalable decryption jailbreak. By injecting an encrypted reasoning trace from a given model into a weaker, and less safeguarded model from the same provider, we force it to decode and output the trace verbatim in plaintext, without ever jailbreaking the more capable model directly. This vulnerability enables four distinct attack vectors. First, it circumvents anti-distillation mechanisms, allowing adversaries to extract a proprietary model's reasoning, as we demonstrate across Anthropic, OpenAI, and Google. Second, it allows for large-scale private data extraction. Developers frequently share session logs publicly, unaware of contents of the encrypted blocks. By decoding 315,320 reasoning blocks scraped from public repositories, we recovered 367 Personally Identifiable Information (PII) artifacts and 182 credentials. Third, it inadvertently reveals hazardous information hidden within the reasoning process, even in cases where the model's final, visible output safely rejects a malicious request. Fourth, attackers can leverage this flaw to execute invisible prompt injections, embedding malicious payloads entirely within encrypted blocks to poison public agentic rollouts. Following responsible disclosure, we propose concrete cryptographic and system-level mitigations to secure client-side reasoning.

Related papers

• DEFUSE: Generalizable Backdoor Defense for Self-Supervised Encoders with Generative Priors [40.98521902229488]
  Self-supervised learning (SSL) encoders are vulnerable to backdoor attacks.<n>DEFUSE is a generalizable backdoor detection framework for SSL encoders.
  arXiv  Detail & Related papers  (2026-08-26T14:24:05Z)
• Fully Homomorphic Encryption on Llama 3 model for privacy preserving LLM inference [1.6542391182229608]
  We integrate the Post-Quantum Cryptography (PQC) based Lattice-based Homomorphic Encryption (HE) main functions in the LLM's inference pipeline to secure some of its layers against data privacy attacks.<n>We demonstrate high text generation accuracies (up to 98%) with reasonable latencies (237 ms) on an i9 CPU, reaching up to 80 tokens per second.
  arXiv  Detail & Related papers  (2026-04-14T00:54:24Z)
• Bag of Tricks for Subverting Reasoning-based Safety Guardrails [62.139297207938036]
  We present a bag of jailbreak methods that subvert the reasoning-based guardrails.<n>Our attacks span white-, gray-, and black-box settings and range from effortless template manipulations to fully automated optimization.
  arXiv  Detail & Related papers  (2025-10-13T16:16:44Z)
• Malice in Agentland: Down the Rabbit Hole of Backdoors in the AI Supply Chain [82.98626829232899]
  Fine-tuning AI agents on data from their own interactions introduces a critical security vulnerability within the AI supply chain.<n>We show that adversaries can easily poison the data collection pipeline to embed hard-to-detect backdoors.
  arXiv  Detail & Related papers  (2025-10-03T12:47:21Z)
• ExtendAttack: Attacking Servers of LRMs via Extending Reasoning [27.205747119390846]
  Large Reasoning Models (LRMs) have demonstrated promising performance in complex tasks.<n>We propose a novel attack method on LRMs termed ExtendAttack to maliciously occupy the resources of servers.<n>We show that ExtendAttack increases the length of the model's response by over 2.5 times for the o3 model on the HumanEval benchmark.
  arXiv  Detail & Related papers  (2025-06-16T17:49:05Z)
• SONNI: Secure Oblivious Neural Network Inference [0.29494468099506893]
  We identify a weakness in this protocol that enables a completely undetectable novel model-stealing attack.<n>This attack works even under multikey encryption that prevents a simple collusion attack to steal model parameters.<n>We propose a mitigation that protects privacy even in the presence of a malicious server and malicious client or model provider.
  arXiv  Detail & Related papers  (2025-04-26T17:23:38Z)
• To Think or Not to Think: Exploring the Unthinking Vulnerability in Large Reasoning Models [56.19026073319406]
  Large Reasoning Models (LRMs) are designed to solve complex tasks by generating explicit reasoning traces before producing final answers.<n>We reveal a critical vulnerability in LRMs -- termed Unthinking -- wherein the thinking process can be bypassed by manipulating special tokens.<n>In this paper, we investigate this vulnerability from both malicious and beneficial perspectives.
  arXiv  Detail & Related papers  (2025-02-16T10:45:56Z)
• Turning Logic Against Itself : Probing Model Defenses Through Contrastive Questions [50.40122190627256]
  We introduce POATE, a novel jailbreak technique that harnesses contrastive reasoning to provoke unethical responses.<n>PoATE crafts semantically opposing intents and integrates them with adversarial templates, steering models toward harmful outputs with remarkable subtlety.<n>To counter this, we propose Intent-Aware CoT and Reverse Thinking CoT, which decompose queries to detect malicious intent and reason in reverse to evaluate and reject harmful responses.
  arXiv  Detail & Related papers  (2025-01-03T15:40:03Z)
• Exploiting Leakage in Password Managers via Injection Attacks [16.120271337898235]
  This work explores injection attacks against password managers. In this setting, the adversary controls their own application client, which they use to "inject" chosen payloads to a victim's client via, for example, sharing credentials with them.
  arXiv  Detail & Related papers  (2024-08-13T17:45:12Z)
• CodeChameleon: Personalized Encryption Framework for Jailbreaking Large Language Models [49.60006012946767]
  We propose CodeChameleon, a novel jailbreak framework based on personalized encryption tactics. We conduct extensive experiments on 7 Large Language Models, achieving state-of-the-art average Attack Success Rate (ASR) Remarkably, our method achieves an 86.6% ASR on GPT-4-1106.
  arXiv  Detail & Related papers  (2024-02-26T16:35:59Z)
• Feature Analysis of Encrypted Malicious Traffic [3.3148826359547514]
  In recent years there has been a dramatic increase in the number of malware attacks that use encrypted HTTP traffic for self-propagation or communication. Antivirus software and firewalls typically will not have access to encryption keys, and therefore direct detection of encrypted data is unlikely to succeed. Previous work has shown that traffic analysis can provide indications of malicious intent, even in cases where the underlying data remains encrypted.
  arXiv  Detail & Related papers  (2023-12-06T12:04:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.