Seed Hijacking of LLM Sampling and Quantum Random Number Defense

• URL: http://arxiv.org/abs/2605.08313v1
• Date: Fri, 08 May 2026 14:17:06 GMT
• Title: Seed Hijacking of LLM Sampling and Quantum Random Number Defense
• Abstract summary: Large language models (LLMs) rely on deterministic pseudorandom number generators (PRNGs) for autoregressive sampling.<n>We present SeedHijack, a backdoor attack that manipulates PRNG outputs to force attacker-specified token selection.<n>In a 540-trial benchmark on GPT-2 (124M), the attack achieves 99.6% exact token injection across 9 sampling configurations.
• Score: 3.278025942906412
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) rely on deterministic pseudorandom number generators (PRNGs) for autoregressive sampling, creating a critical supply-chain attack surface overlooked by existing defenses. We present SeedHijack, a backdoor attack that manipulates PRNG outputs to force attacker-specified token selection without altering model logits. In a 540-trial benchmark on GPT-2 (124M), the attack achieves 99.6% exact token injection across 9 sampling configurations; it reaches 100% success on four aligned models (1.5B-7B, RLHF/SFT/reasoning distillation) and bypasses all alignment methods tested in this work. We further propose a defense based on a hardware quantum random number generator (QRNG), which neutralizes the attack in our evaluated threat model with negligible median overhead (+0.6% latency, +7.7 MB memory). Our work identifies a critical sampling-layer vulnerability and provides a practical, deployable QRNG-based defense.

Related papers

• DiffusionHijack: Supply-Chain PRNG Backdoor Attack on Diffusion Models and Quantum Random Number Defense [0.6455316503462029]
  Diffusion models depend on pseudo-random number generators (PRNGs) for latent noise sampling.<n>We present DiffusionHijack, a supply-chain backdoor attack that hijacks the PRNG to deterministically control generated images.
  arXiv  Detail & Related papers  (2026-05-13T07:34:04Z)
• Semi-Device-Independent Quantum Random Number Generator Resistant to General Attacks [6.918556765582295]
  Quantum random number generators (QRNGs) produce true random numbers based on the inherent randomness of quantum theory.<n>In this paper, we propose a semi-DI QRNG that resists general attacks while accounting for finite-size effects.<n>Our work offers a promising approach to achieve both the robust security and high generation rate with a simple experimental setup.
  arXiv  Detail & Related papers  (2026-02-06T03:47:12Z)
• Sampling-aware Adversarial Attacks Against Large Language Models [52.30089653615172]
  Existing adversarial attacks typically target harmful responses in single-point greedy generations.<n>We show that for the goal of eliciting harmful responses, repeated sampling of model outputs during the attack prompt optimization.<n>We show that integrating sampling into existing attacks boosts success rates by up to 37% and improves efficiency by up to two orders of magnitude.
  arXiv  Detail & Related papers  (2025-07-06T16:13:33Z)
• Uncertainty is Fragile: Manipulating Uncertainty in Large Language Models [79.76293901420146]
  Large Language Models (LLMs) are employed across various high-stakes domains, where the reliability of their outputs is crucial. Our research investigates the fragility of uncertainty estimation and explores potential attacks. We demonstrate that an attacker can embed a backdoor in LLMs, which, when activated by a specific trigger in the input, manipulates the model's uncertainty without affecting the final output.
  arXiv  Detail & Related papers  (2024-07-15T23:41:11Z)
• Stealthy Backdoor Attack via Confidence-driven Sampling [49.72680157684523]
  Backdoor attacks aim to surreptitiously insert malicious triggers into DNN models, granting unauthorized control during testing scenarios.<n>Existing methods lack robustness against defense strategies and predominantly focus on enhancing trigger stealthiness while randomly selecting poisoned samples.<n>We introduce a straightforward yet highly effective sampling methodology that leverages confidence scores. Specifically, it selects samples with lower confidence scores, significantly increasing the challenge for defenders in identifying and countering these attacks.
  arXiv  Detail & Related papers  (2023-10-08T18:57:36Z)
• One-bit Flip is All You Need: When Bit-flip Attack Meets Model Training [54.622474306336635]
  A new weight modification attack called bit flip attack (BFA) was proposed, which exploits memory fault inject techniques. We propose a training-assisted bit flip attack, in which the adversary is involved in the training stage to build a high-risk model to release.
  arXiv  Detail & Related papers  (2023-08-12T09:34:43Z)
• Machine Learning needs Better Randomness Standards: Randomised Smoothing and PRNG-based attacks [14.496582479888765]
  We consider whether attackers can compromise an machine learning system using only the randomness on which they commonly rely. We demonstrate an entirely novel attack, where an attacker backdoors the supplied randomness to falsely certify either an overestimate or an underestimate of robustness for up to 81 times. We advocate updating the NIST guidelines on random number testing to make them more appropriate for safety-critical and security-critical machine-learning applications.
  arXiv  Detail & Related papers  (2023-06-24T19:50:08Z)
• MultiRobustBench: Benchmarking Robustness Against Multiple Attacks [86.70417016955459]
  We present the first unified framework for considering multiple attacks against machine learning (ML) models. Our framework is able to model different levels of learner's knowledge about the test-time adversary. We evaluate the performance of 16 defended models for robustness against a set of 9 different attack types.
  arXiv  Detail & Related papers  (2023-02-21T20:26:39Z)
• Source-independent quantum random number generator against tailored detector blinding attacks [6.86599501487994]
  We propose a quantum random number generation protocol that addresses source vulnerability and ferocious detector blinding attacks. We experimentally demonstrate the ability of our protocol to generate random numbers for two-dimensional measurement with a generation speed of 0.1 bit per pulse.
  arXiv  Detail & Related papers  (2022-04-26T08:47:37Z)
• Defence against adversarial attacks using classical and quantum-enhanced Boltzmann machines [64.62510681492994]
  generative models attempt to learn the distribution underlying a dataset, making them inherently more robust to small perturbations. We find improvements ranging from 5% to 72% against attacks with Boltzmann machines on the MNIST dataset.
  arXiv  Detail & Related papers  (2020-12-21T19:00:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.