Sockpuppetting: Jailbreaking LLMs Without Optimization Through Output Prefix Injection

• URL: http://arxiv.org/abs/2601.13359v1
• Date: Mon, 19 Jan 2026 19:53:48 GMT
• Title: Sockpuppetting: Jailbreaking LLMs Without Optimization Through Output Prefix Injection
• Abstract summary: "sockpuppetting" is a simple method for jailbreaking open-weight language models.<n>It achieves up to 80% higher attack success rate (ASR) than GCG on Qwen3-8B in per-prompt comparisons.
• Score: 2.8329969194317
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As open-weight large language models (LLMs) increase in capabilities, safeguarding them against malicious prompts and understanding possible attack vectors becomes ever more important. While automated jailbreaking methods like GCG [Zou et al., 2023] remain effective, they often require substantial computational resources and specific expertise. We introduce "sockpuppetting'', a simple method for jailbreaking open-weight LLMs by inserting an acceptance sequence (e.g., "Sure, here is how to...'') at the start of a model's output and allowing it to complete the response. Requiring only a single line of code and no optimization, sockpuppetting achieves up to 80% higher attack success rate (ASR) than GCG on Qwen3-8B in per-prompt comparisons. We also explore a hybrid approach that optimizes the adversarial suffix within the assistant message block rather than the user prompt, increasing ASR by 64% over GCG on Llama-3.1-8B in a prompt-agnostic setting. The results establish sockpuppetting as an effective low-cost attack accessible to unsophisticated adversaries, highlighting the need for defences against output-prefix injection in open-weight models.

Related papers

• SlotGCG: Exploiting the Positional Vulnerability in LLMs for Jailbreak Attacks [12.686897698255406]
  We introduce the textitVulnerable Slot Score (VSS) to quantify the positional vulnerability to jailbreaking.<n>We then propose SlotGCG, which selects the most vulnerable slots for insertion, and runs a targeted optimization attack at those slots.<n>Our approach provides a position-search mechanism that is attack-agnostic and can be plugged into any optimization-based attack, adding only 200ms of preprocessing time.
  arXiv  Detail & Related papers  (2026-06-04T02:31:29Z)
• TrapSuffix: Proactive Defense Against Adversarial Suffixes in Jailbreaking [52.72486831074384]
  Suffix-based jailbreak attacks append an adversarial suffix, i.e., a short token sequence, to steer aligned LLMs into unsafe outputs.<n>We propose TrapSuffix, a lightweight fine-tuning approach that injects trap-aligned behaviors into the base model without changing the inference pipeline.<n>Across diverse suffix-based jailbreak settings, TrapSuffix reduces the average attack success rate to below 0.01 percent and achieves an average tracing success rate of 87.9 percent.
  arXiv  Detail & Related papers  (2026-02-06T11:43:56Z)
• RECAP: A Resource-Efficient Method for Adversarial Prompting in Large Language Models [0.0]
  This paper introduces a resource-efficient adversarial prompting approach that eliminates the need for retraining by matching new prompts to a database of pre-trained adversarial prompts.<n>By retrieving semantically similar successful adversarial prompts, the proposed method achieves competitive attack success rates with significantly reduced computational cost.
  arXiv  Detail & Related papers  (2026-01-20T06:01:02Z)
• Bag of Tricks for Subverting Reasoning-based Safety Guardrails [62.139297207938036]
  We present a bag of jailbreak methods that subvert the reasoning-based guardrails.<n>Our attacks span white-, gray-, and black-box settings and range from effortless template manipulations to fully automated optimization.
  arXiv  Detail & Related papers  (2025-10-13T16:16:44Z)
• Revisiting Backdoor Attacks on LLMs: A Stealthy and Practical Poisoning Framework via Harmless Inputs [54.90315421117162]
  We propose a novel poisoning method via completely harmless data.<n>Inspired by the causal reasoning in auto-regressive LLMs, we aim to establish robust associations between triggers and an affirmative response prefix.<n>We observe an interesting resistance phenomenon where the LLM initially appears to agree but subsequently refuses to answer.
  arXiv  Detail & Related papers  (2025-05-23T08:13:59Z)
• REINFORCE Adversarial Attacks on Large Language Models: An Adaptive, Distributional, and Semantic Objective [57.57786477441956]
  We propose an adaptive and semantic optimization problem over the population of responses.<n>Our objective doubles the attack success rate (ASR) on Llama3 and increases the ASR from 2% to 50% with circuit breaker defense.
  arXiv  Detail & Related papers  (2025-02-24T15:34:48Z)
• Enhancing Adversarial Attacks through Chain of Thought [0.0]
  gradient-based adversarial attacks are particularly effective against aligned large language models (LLMs) This paper proposes enhancing the universality of adversarial attacks by integrating CoT prompts with the greedy coordinate gradient (GCG) technique.
  arXiv  Detail & Related papers  (2024-10-29T06:54:00Z)
• Faster-GCG: Efficient Discrete Optimization Jailbreak Attacks against Aligned Large Language Models [16.938267820586024]
  We propose Faster-GCG, an efficient adversarial jailbreak method by delving deep into the design of GCG. Experiments demonstrate that Faster-GCG can surpass the original GCG with only 1/10 of the computational cost.
  arXiv  Detail & Related papers  (2024-10-20T11:27:41Z)
• Improved Techniques for Optimization-Based Jailbreaking on Large Language Models [78.32176751215073]
  Greedy Coordinate Gradient (GCG) attack's success has led to a growing interest in the study of optimization-based jailbreaking techniques. We present several improved (empirical) techniques for optimization-based jailbreaks like GCG. The results demonstrate that our improved techniques can help GCG outperform state-of-the-art jailbreaking attacks and achieve nearly 100% attack success rate.
  arXiv  Detail & Related papers  (2024-05-31T17:07:15Z)
• Improved Generation of Adversarial Examples Against Safety-aligned LLMs [72.38072942860309]
  Adversarial prompts generated using gradient-based methods exhibit outstanding performance in performing automatic jailbreak attacks against safety-aligned LLMs. In this paper, we explore a new perspective on this problem, suggesting that it can be alleviated by leveraging innovations inspired in transfer-based attacks. We show that 87% of the query-specific adversarial suffixes generated by the developed combination can induce Llama-2-7B-Chat to produce the output that exactly matches the target string on AdvBench.
  arXiv  Detail & Related papers  (2024-05-28T06:10:12Z)
• AmpleGCG: Learning a Universal and Transferable Generative Model of Adversarial Suffixes for Jailbreaking Both Open and Closed LLMs [11.094625711201648]
  GCGcitepzou2023universal proposes a discrete token optimization algorithm and selects the single suffix with the lowest loss to successfully jailbreak aligned LLMs. We utilize successful suffixes as training data to learn a generative model, named AmpleGCG, which captures the distribution of adversarial suffixes given a harmful query. A AmpleGCG model can generate 200 adversarial suffixes for one harmful query in only 4 seconds, rendering it more challenging to defend.
  arXiv  Detail & Related papers  (2024-04-11T17:05:50Z)
• PAL: Proxy-Guided Black-Box Attack on Large Language Models [55.57987172146731]
  Large Language Models (LLMs) have surged in popularity in recent months, but they have demonstrated capabilities to generate harmful content when manipulated. We introduce the Proxy-Guided Attack on LLMs (PAL), the first optimization-based attack on LLMs in a black-box query-only setting. Our attack achieves 84% attack success rate (ASR) on GPT-3.5-Turbo and 48% on Llama-2-7B, compared to 4% for the current state of the art.
  arXiv  Detail & Related papers  (2024-02-15T02:54:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.