Hail to the Thief: Exploring Attacks and Defenses in Decentralised GRPO

• URL: http://arxiv.org/abs/2511.09780v1
• Date: Fri, 14 Nov 2025 01:09:30 GMT
• Title: Hail to the Thief: Exploring Attacks and Defenses in Decentralised GRPO
• Abstract summary: We present the first adversarial attack in decentralised GRPO.<n>We show that adversarial attacks can easily poison the benign nodes post-training, achieving attack success rates up to 100% in as few as 50 iterations.<n>We propose two ways to defend against these attacks, depending on whether all users train the same model or different models.
• Score: 0.9048611509540079
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Group Relative Policy Optimization (GRPO) has demonstrated great utilization in post-training of Large Language Models (LLMs). In GRPO, prompts are answered by the model and, through reinforcement learning, preferred completions are learnt. Owing to the small communication volume, GRPO is inherently suitable for decentralised training as the prompts can be concurrently answered by multiple nodes and then exchanged in the forms of strings. In this work, we present the first adversarial attack in decentralised GRPO. We demonstrate that malicious parties can poison such systems by injecting arbitrary malicious tokens in benign models in both out-of-context and in-context attacks. Using empirical examples of math and coding tasks, we show that adversarial attacks can easily poison the benign nodes, polluting their local LLM post-training, achieving attack success rates up to 100% in as few as 50 iterations. We propose two ways to defend against these attacks, depending on whether all users train the same model or different models. We show that these defenses can achieve stop rates of up to 100%, making the attack impossible.

Related papers

• Defending against Adaptive Prompt Injection Attacks via Reasoning-enabled Task Alignment [25.752599132396437]
  Indirect prompt injection attacks hijack LLM-based agents by embedding malicious instructions in third-party data that the agent retrieves during task execution.<n>Existing defenses report near-zero attack success rate on static benchmarks, yet recent adaptive evaluations show that these results collapse once the attacker is allowed to optimize against the deployed defense.<n>We propose RETA, a training-based method that grounds defense decisions on the user tasks rather than attacker-controlled data.
  arXiv  Detail & Related papers  (2026-06-13T19:15:44Z)
• Learning to Attack and Defend: Adaptive Red Teaming of Language Models via GRPO [2.714570887733442]
  Adv GRPO is a co-training framework that makes attacker-defender optimization using dense multi-channel rewards.<n>We show that our method can produce highly effective and transferable attacks and that co-trained defenders outperform baselines on safety benchmarks.
  arXiv  Detail & Related papers  (2026-06-08T16:21:36Z)
• MAGIC: A Co-Evolving Attacker-Defender Adversarial Game for Robust LLM Safety [28.246225272659917]
  This paper introduces textbfMAGIC, a novel multi-turn multi-agent reinforcement learning framework.<n>It formulates Large Language Models safety alignment as an adversarial asymmetric game.<n>Our framework demonstrates superior defense success rates without compromising the helpfulness of the model.
  arXiv  Detail & Related papers  (2026-02-02T02:12:28Z)
• P2P: A Poison-to-Poison Remedy for Reliable Backdoor Defense in LLMs [49.908234151374785]
  During fine-tuning, large language models (LLMs) are increasingly vulnerable to data-poisoning backdoor attacks.<n>We propose Poison-to-Poison (P2P), a general and effective backdoor defense algorithm.<n>We show that P2P can neutralize malicious backdoors while preserving task performance.
  arXiv  Detail & Related papers  (2025-10-06T05:45:23Z)
• An Interpretable N-gram Perplexity Threat Model for Large Language Model Jailbreaks [87.64278063236847]
  In this work, we propose a unified threat model for the principled comparison of jailbreak attacks.<n>Our threat model checks if a given jailbreak is likely to occur in the distribution of text.<n>We adapt popular attacks to this threat model, and, for the first time, benchmark these attacks on equal footing with it.
  arXiv  Detail & Related papers  (2024-10-21T17:27:01Z)
• Learning diverse attacks on large language models for robust red-teaming and safety tuning [126.32539952157083]
  Red-teaming, or identifying prompts that elicit harmful responses, is a critical step in ensuring the safe deployment of large language models.<n>We show that even with explicit regularization to favor novelty and diversity, existing approaches suffer from mode collapse or fail to generate effective attacks.<n>We propose to use GFlowNet fine-tuning, followed by a secondary smoothing phase, to train the attacker model to generate diverse and effective attack prompts.
  arXiv  Detail & Related papers  (2024-05-28T19:16:17Z)
• GenFighter: A Generative and Evolutive Textual Attack Removal [6.044610337297754]
  Adrial attacks pose significant challenges to deep neural networks (DNNs) such as Transformer models in natural language processing (NLP) This paper introduces a novel defense strategy, called GenFighter, which enhances adversarial robustness by learning and reasoning on the training classification distribution. We show that GenFighter outperforms state-of-the-art defenses in accuracy under attack and attack success rate metrics.
  arXiv  Detail & Related papers  (2024-04-17T16:32:13Z)
• Universal Vulnerabilities in Large Language Models: Backdoor Attacks for In-context Learning [14.011140902511135]
  In-context learning, a paradigm bridging the gap between pre-training and fine-tuning, has demonstrated high efficacy in several NLP tasks. Despite being widely applied, in-context learning is vulnerable to malicious attacks. We design a new backdoor attack method, named ICLAttack, to target large language models based on in-context learning.
  arXiv  Detail & Related papers  (2024-01-11T14:38:19Z)
• Large Language Models Are Better Adversaries: Exploring Generative Clean-Label Backdoor Attacks Against Text Classifiers [25.94356063000699]
  Backdoor attacks manipulate model predictions by inserting innocuous triggers into training and test data. We focus on more realistic and more challenging clean-label attacks where the adversarial training examples are correctly labeled. Our attack, LLMBkd, leverages language models to automatically insert diverse style-based triggers into texts.
  arXiv  Detail & Related papers  (2023-10-28T06:11:07Z)
• Backdoor Attacks in Peer-to-Peer Federated Learning [11.235386862864397]
  Peer-to-Peer Federated Learning (P2PFL) offer advantages in terms of both privacy and reliability. We propose new backdoor attacks for P2PFL that leverage structural graph properties to select the malicious nodes, and achieve high attack success.
  arXiv  Detail & Related papers  (2023-01-23T21:49:28Z)
• Universal Adversarial Training with Class-Wise Perturbations [78.05383266222285]
  adversarial training is the most widely used method for defending against adversarial attacks. In this work, we find that a UAP does not attack all classes equally. We improve the SOTA UAT by proposing to utilize class-wise UAPs during adversarial training.
  arXiv  Detail & Related papers  (2021-04-07T09:05:49Z)
• Adversarial Example Games [51.92698856933169]
  Adrial Example Games (AEG) is a framework that models the crafting of adversarial examples. AEG provides a new way to design adversarial examples by adversarially training a generator and aversa from a given hypothesis class. We demonstrate the efficacy of AEG on the MNIST and CIFAR-10 datasets.
  arXiv  Detail & Related papers  (2020-07-01T19:47:23Z)
• A Self-supervised Approach for Adversarial Robustness [105.88250594033053]
  Adversarial examples can cause catastrophic mistakes in Deep Neural Network (DNNs) based vision systems. This paper proposes a self-supervised adversarial training mechanism in the input space. It provides significant robustness against the textbfunseen adversarial attacks.
  arXiv  Detail & Related papers  (2020-06-08T20:42:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.