When Speculation Spills Secrets: Side Channels via Speculative Decoding In LLMs

• URL: http://arxiv.org/abs/2411.01076v3
• Date: Fri, 26 Sep 2025 23:28:00 GMT
• Title: When Speculation Spills Secrets: Side Channels via Speculative Decoding In LLMs
• Abstract summary: In this work, we reveal a new side-channel whereby input-dependent patterns of correct and incorrect speculations can be inferred by monitoring per-iteration token counts or packet sizes.<n>An adversary observing these patterns can fingerprint user queries with >90% accuracy across four speculative-decoding schemes.<n>We propose and evaluate a suite of mitigations, including packet padding and iteration-wise token aggregation.
• Score: 2.305309289946973
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deployed large language models (LLMs) often rely on speculative decoding, a technique that generates and verifies multiple candidate tokens in parallel, to improve throughput and latency. In this work, we reveal a new side-channel whereby input-dependent patterns of correct and incorrect speculations can be inferred by monitoring per-iteration token counts or packet sizes.We demonstrate that an adversary observing these patterns can fingerprint user queries with >90% accuracy across four speculative-decoding schemes, REST (100\%), LADE (up to 92%), BiLD (up to 95%), and EAGLE (up to 77.6%) and leak confidential datastore contents used for prediction at rates exceeding 25 tokens/sec. We evaluate the side-channel attacks in both research prototypes as well as the production-grade vLLM serving framework. To defend against these, we propose and evaluate a suite of mitigations, including packet padding and iteration-wise token aggregation.

Related papers

• Ethereum NFT Smart Contracts: Knowledge-Guided Vulnerability Detection with LLM and Code Slicing [2.0706429677946563]
  We present a vulnerability-detection method that combines vulnerability-focused code slicing, an ERC-721-oriented knowledge base, and constrained DeepSeek analysis.<n>On 450 NFT contract samples, the full configuration produced 437 positive labels, corresponding to a reported positive-label rate of 97.1%.<n>Removing the external knowledge base reduced this rate to 87.11%, while analyzing complete contracts without the knowledge base reduced it to 73.78%.
  arXiv  Detail & Related papers  (2026-07-24T05:10:26Z)
• Constrained Decoding for Diffusion Language Models via Efficient Inference over Finite Automata [57.27430779838529]
  We present an exact and tractable algorithm for sampling from the constrained mean-field posterior under any constraint expressible as a finite automaton.<n>The approach guarantees constraint satisfaction by construction, supports both greedy and sampling-based decoding, and is compatible with parallel and block-wise decoding.<n> Empirical evaluations on Dream-7B and LLaDA-8 show substantial accuracy gains across various tasks.
  arXiv  Detail & Related papers  (2026-07-08T05:48:57Z)
• Compliance-Scored Best-of-N Guardrail Orchestration for Multimodal Document Generation in Payments Dispute Defense [0.0]
  High-stakes enterprise document generation includes financial dispute narratives, compliance notices, and audit summaries.<n>Prior to a unified guardrail layer, production systems often stitched together separate PII redaction, content moderation, and format validation steps.<n>We present a guardrail orchestration layer for text and image inputs that couples multi-candidate generation with an explicit compliance score used for early exit.
  arXiv  Detail & Related papers  (2026-06-01T00:24:30Z)
• WISV: Wireless-Informed Semantic Verification for Distributed Speculative Decoding in Device-Edge LLM Inference [56.297697169678095]
  WISV (Wireless-Informed Semantic Verification) is a novel distributed speculative decoding framework.<n>We show that WISV achieves up to a 60.8% increase in accepted length, a 37.3% reduction in interaction rounds, and a 31.4% improvement in end-to-end latency.<n>We validate WISV on a hardware testbed comprising an NVIDIA Jetson AGX Orin and an A40-equipped server.
  arXiv  Detail & Related papers  (2026-04-20T01:29:56Z)
• Towards Verifiable AI with Lightweight Cryptographic Proofs of Inference [3.3323431541048385]
  We present a verification framework and protocol that replaces full proofs with a lightweight, sampling-based approach.<n>We formalize the conditions under which trace separation between functionally dissimilar models can be leveraged to argue the security of verifiable inference protocols.<n>Our approach reduces proving times by several orders of magnitude compared to state-of-the-art cryptographic proof systems.
  arXiv  Detail & Related papers  (2026-03-19T15:24:27Z)
• Taint-Based Code Slicing for LLMs-based Malicious NPM Package Detection [2.398400814870029]
  This paper introduces a novel framework that leverages code slicing techniques for an LLM-based malicious package detection task.<n>We propose a specialized taintbased slicing technique for npm packages, augmented by a backtracking mechanism.<n>An evaluation on a dataset of more than 5000 malicious and benign npm packages demonstrates that our approach isolates security-relevant code, reducing input volume by over 99%.
  arXiv  Detail & Related papers  (2025-12-13T12:56:03Z)
• SWAP: Towards Copyright Auditing of Soft Prompts via Sequential Watermarking [58.475471437150674]
  We propose sequential watermarking for soft prompts (SWAP)<n>SWAP encodes watermarks through a specific order of defender-specified out-of-distribution classes.<n>Experiments on 11 datasets demonstrate SWAP's effectiveness, harmlessness, and robustness against potential adaptive attacks.
  arXiv  Detail & Related papers  (2025-11-05T13:48:48Z)
• dParallel: Learnable Parallel Decoding for dLLMs [77.24184219948337]
  Diffusion large language models (dLLMs) offer parallel token prediction and lower inference latency.<n>Existing open-source models still require nearly token-length decoding steps to ensure performance.<n>We introduce dParallel, a simple and effective method that unlocks the inherent parallelism of dLLMs for fast sampling.
  arXiv  Detail & Related papers  (2025-09-30T16:32:52Z)
• PromptCOS: Towards System Prompt Copyright Auditing for LLMs via Content-level Output Similarity [61.793486262641345]
  We propose PromptCOS, a method for auditing prompt copyright based on content-level output similarity.<n>It embeds watermarks by optimizing the prompt while simultaneously co-optimizing a special verification query and content-level signal marks.<n>For copyright verification, PromptCOS identifies unauthorized usage by comparing the similarity between the suspicious output and the signal mark.
  arXiv  Detail & Related papers  (2025-09-03T08:19:40Z)
• Conformal Prediction for Privacy-Preserving Machine Learning [83.88591755871734]
  Using AES-encrypted variants of the MNIST dataset, we demonstrate that Conformal Prediction methods remain effective even when applied directly in the encrypted domain.<n>Our work sets a foundation for principled uncertainty quantification in secure, privacy-aware learning systems.
  arXiv  Detail & Related papers  (2025-07-13T15:29:14Z)
• Detecting Hard-Coded Credentials in Software Repositories via LLMs [0.0]
  Software developers frequently hard-code credentials such as passwords, generic secrets, private keys, and generic tokens in software repositories.<n>These credentials create attack surfaces exploitable by a potential adversary to conduct malicious exploits such as backdoor attacks.<n>Recent detection efforts utilize embedding models to vectorize textual credentials before passing them to classifiers for predictions.<n>Our model outperforms the current state-of-the-art by 13% in F1 measure on the benchmark dataset.
  arXiv  Detail & Related papers  (2025-06-16T04:33:48Z)
• Think Before You Accept: Semantic Reflective Verification for Faster Speculative Decoding [48.52389201779425]
  Speculative decoding accelerates inference by generating multiple draft tokens using a lightweight model and verifying them in parallel.<n>Existing verification methods rely heavily on distributional consistency while overlooking semantic correctness.<n>We propose Reflective Verification, a training-free and semantics-aware approach that achieves a better trade-off between correctness and efficiency.
  arXiv  Detail & Related papers  (2025-05-24T10:26:27Z)
• Prompt, Divide, and Conquer: Bypassing Large Language Model Safety Filters via Segmented and Distributed Prompt Processing [1.4201040196058878]
  Large Language Models (LLMs) have transformed task automation and content generation across various domains.<n>We introduce a novel jailbreaking framework that employs distributed prompt processing combined with iterative refinements to bypass safety measures.<n>Tested on 500 malicious prompts across 10 cybersecurity categories, the framework achieves a 73.2% Success Rate (SR) in generating malicious code.
  arXiv  Detail & Related papers  (2025-03-27T15:19:55Z)
• Judge Decoding: Faster Speculative Sampling Requires Going Beyond Model Alignment [25.988070517700848]
  Speculative decoding has been proposed as a technique to accelerate autoregressive generation.<n>We show that even powerful draft models such as GPT-4o, as well as human text cannot achieve high acceptance rates.<n>We ask the following question: Can we adapt verification to recognize correct, but non-aligned replies?
  arXiv  Detail & Related papers  (2025-01-31T17:09:53Z)
• FIRP: Faster LLM inference via future intermediate representation prediction [54.897493351694195]
  FIRP generates multiple tokens instead of one at each decoding step. We conduct extensive experiments, showing a speedup ratio of 1.9x-3x in several models and datasets.
  arXiv  Detail & Related papers  (2024-10-27T15:53:49Z)
• Adaptive Pre-training Data Detection for Large Language Models via Surprising Tokens [1.2549198550400134]
  Large language models (LLMs) are extensively used, but there are concerns regarding privacy, security, and copyright due to their opaque training data. Current solutions to this problem leverage techniques explored in machine learning privacy such as Membership Inference Attacks (MIAs) We propose an adaptive pre-training data detection method which alleviates this reliance and effectively amplify the identification.
  arXiv  Detail & Related papers  (2024-07-30T23:43:59Z)
• Private prediction for large-scale synthetic text generation [28.488459921169905]
  We present an approach for generating differentially private synthetic text using large language models (LLMs) In the private prediction framework, we only require the output synthetic data to satisfy differential privacy guarantees.
  arXiv  Detail & Related papers  (2024-07-16T18:28:40Z)
• Lazy Layers to Make Fine-Tuned Diffusion Models More Traceable [70.77600345240867]
  A novel arbitrary-in-arbitrary-out (AIAO) strategy makes watermarks resilient to fine-tuning-based removal. Unlike the existing methods of designing a backdoor for the input/output space of diffusion models, in our method, we propose to embed the backdoor into the feature space of sampled subpaths. Our empirical studies on the MS-COCO, AFHQ, LSUN, CUB-200, and DreamBooth datasets confirm the robustness of AIAO.
  arXiv  Detail & Related papers  (2024-05-01T12:03:39Z)
• Token-Level Adversarial Prompt Detection Based on Perplexity Measures and Contextual Information [67.78183175605761]
  Large Language Models are susceptible to adversarial prompt attacks. This vulnerability underscores a significant concern regarding the robustness and reliability of LLMs. We introduce a novel approach to detecting adversarial prompts at a token level.
  arXiv  Detail & Related papers  (2023-11-20T03:17:21Z)
• PrivacyMind: Large Language Models Can Be Contextual Privacy Protection Learners [81.571305826793]
  We introduce Contextual Privacy Protection Language Models (PrivacyMind) Our work offers a theoretical analysis for model design and benchmarks various techniques. In particular, instruction tuning with both positive and negative examples stands out as a promising method.
  arXiv  Detail & Related papers  (2023-10-03T22:37:01Z)
• Differentially Private Secure Multiplication: Hiding Information in the Rubble of Noise [7.767656876470667]
  We consider the problem of private distributed multi-party multiplication. It is well-established that Shamir secret-sharing coding strategies can enable perfect information-theoretic privacy in distributed computation.
  arXiv  Detail & Related papers  (2023-09-28T02:13:13Z)
• Tight Auditing of Differentially Private Machine Learning [77.38590306275877]
  For private machine learning, existing auditing mechanisms are tight. They only give tight estimates under implausible worst-case assumptions. We design an improved auditing scheme that yields tight privacy estimates for natural (not adversarially crafted) datasets.
  arXiv  Detail & Related papers  (2023-02-15T21:40:33Z)
• Planting and Mitigating Memorized Content in Predictive-Text Language Models [11.911353678499008]
  Language models are widely deployed to provide automatic text completion services in user products. Recent research has revealed that language models bear considerable risk of memorizing private training data. In this study, we test the efficacy of a range of privacy-preserving techniques to mitigate unintended memorization of sensitive user text.
  arXiv  Detail & Related papers  (2022-12-16T17:57:14Z)
• Pre-trained Encoders in Self-Supervised Learning Improve Secure and Privacy-preserving Supervised Learning [63.45532264721498]
  Self-supervised learning is an emerging technique to pre-train encoders using unlabeled data. We perform first systematic, principled measurement study to understand whether and when a pretrained encoder can address the limitations of secure or privacy-preserving supervised learning algorithms.
  arXiv  Detail & Related papers  (2022-12-06T21:35:35Z)
• Defending against Reconstruction Attacks with R\'enyi Differential Privacy [72.1188520352079]
  Reconstruction attacks allow an adversary to regenerate data samples of the training set using access to only a trained model. Differential privacy is a known solution to such attacks, but is often used with a relatively large privacy budget. We show that, for a same mechanism, we can derive privacy guarantees for reconstruction attacks that are better than the traditional ones from the literature.
  arXiv  Detail & Related papers  (2022-02-15T18:09:30Z)
• Detecting DeFi Securities Violations from Token Smart Contract Code [0.4263043028086136]
  Decentralized Finance (DeFi) is a system of financial products and services built and delivered through smart contracts on various blockchains. This study aims to uncover whether we can identify DeFi projects potentially engaging in securities violations based on their tokens' smart contract code.
  arXiv  Detail & Related papers  (2021-12-06T01:44:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.