Plan2Cleanse: Test-Time Backdoor Defense via Monte-Carlo Planning in Deep Reinforcement Learning

• URL: http://arxiv.org/abs/2605.09638v1
• Date: Sun, 10 May 2026 16:34:16 GMT
• Title: Plan2Cleanse: Test-Time Backdoor Defense via Monte-Carlo Planning in Deep Reinforcement Learning
• Abstract summary: Plan2Cleanse is a test-time detection and mitigation framework.<n>It adapts Monte Carlo Tree Search to efficiently identify and neutralize RL backdoor attacks.<n>Plan2Cleanse achieves substantial improvements in trigger detection success rates.
• Score: 12.26506262764069
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Ensuring the security of reinforcement learning (RL) models is critical, particularly when they are trained by third parties and deployed in real-world systems. Attackers can implant backdoors into these models, causing them to behave normally under typical conditions, but execute malicious behaviors when specific triggers are activated. In this work, we propose Plan2Cleanse, a test-time detection and mitigation framework that adapts Monte Carlo Tree Search to efficiently identify and neutralize RL backdoor attacks without requiring model retraining. Our approach recasts backdoor detection as a planning problem, enabling systematic exploration of temporally extended trigger sequences while maintaining black-box access to the target policy. By leveraging the detection results, Plan2Cleanse can further achieve efficient mitigation through tree-search preventive replanning. We evaluated our method in competitive MuJoCo environments, simulated O-RAN wireless networks, and Atari games. Plan2Cleanse achieves substantial improvements, increasing trigger detection success rates by more than 61.4 percentage points in stealthy O-RAN scenarios and improving win rates from 35\% to 53\% in competitive Humanoid environments. These results demonstrate the effectiveness of our test-time defense approach and highlight the importance of proactive defenses against backdoor threats in RL deployments. Our implementation is publicly available at https://github.com/rl-bandits-lab/RL-Backdoor.

Related papers

• BehaviorGuard: Online Backdoor Defense for Deep Reinforcement Learning [32.53874142863277]
  Backdoor attacks pose a serious threat to deep reinforcement learning (DRL)<n>Current defenses rely on reward anomalies to reverse-engineer triggers and model finetuning to remove backdoors.<n>We propose BehaviorGuard, an online behavior-based backdoor detection and mitigation framework for DRL.
  arXiv  Detail & Related papers  (2026-05-07T10:21:14Z)
• Injection, Attack and Erasure: Revocable Backdoor Attacks via Machine Unlearning [2.1896295740048894]
  We introduce the first paradigm of revocable backdoor attacks, where the backdoor can be proactively and thoroughly removed after the attack objective is achieved.<n>This work opens a new direction for backdoor attack research and presents new challenges for the security of machine learning systems.
  arXiv  Detail & Related papers  (2025-10-15T09:09:43Z)
• TooBadRL: Trigger Optimization to Boost Effectiveness of Backdoor Attacks on Deep Reinforcement Learning [38.79063331759597]
  TooBadRL is a framework to systematically optimize DRL backdoor triggers along three critical axes, i.e., temporal, spatial, and magnitude.<n>We show that TooBadRL significantly improves attack success rates, while ensuring minimal degradation of normal task performance.
  arXiv  Detail & Related papers  (2025-06-11T09:50:17Z)
• Neural Antidote: Class-Wise Prompt Tuning for Purifying Backdoors in CLIP [51.04452017089568]
  Class-wise Backdoor Prompt Tuning (CBPT) is an efficient and effective defense mechanism that operates on text prompts to indirectly purify CLIP.<n>CBPT significantly mitigates backdoor threats while preserving model utility.
  arXiv  Detail & Related papers  (2025-02-26T16:25:15Z)
• Efficient Backdoor Defense in Multimodal Contrastive Learning: A Token-Level Unlearning Method for Mitigating Threats [52.94388672185062]
  We propose an efficient defense mechanism against backdoor threats using a concept known as machine unlearning. This entails strategically creating a small set of poisoned samples to aid the model's rapid unlearning of backdoor vulnerabilities. In the backdoor unlearning process, we present a novel token-based portion unlearning training regime.
  arXiv  Detail & Related papers  (2024-09-29T02:55:38Z)
• SleeperNets: Universal Backdoor Poisoning Attacks Against Reinforcement Learning Agents [16.350898218047405]
  Reinforcement learning (RL) is an actively growing field that is seeing increased usage in real-world, safety-critical applications. In this work we explore a particularly stealthy form of training-time attacks against RL -- backdoor poisoning. We formulate a novel poisoning attack framework which interlinks the adversary's objectives with those of finding an optimal policy.
  arXiv  Detail & Related papers  (2024-05-30T23:31:25Z)
• BadRL: Sparse Targeted Backdoor Attack Against Reinforcement Learning [37.19070609394519]
  Backdoor attacks in reinforcement learning (RL) have previously employed intense attack strategies to ensure attack success. In this work, we propose a novel approach, BadRL, which focuses on conducting highly sparse backdoor poisoning efforts during training and testing. Our algorithm, BadRL, strategically chooses state observations with high attack values to inject triggers during training and testing, thereby reducing the chances of detection.
  arXiv  Detail & Related papers  (2023-12-19T20:29:29Z)
• BadCLIP: Dual-Embedding Guided Backdoor Attack on Multimodal Contrastive Learning [85.2564206440109]
  This paper reveals the threats in this practical scenario that backdoor attacks can remain effective even after defenses. We introduce the emphtoolns attack, which is resistant to backdoor detection and model fine-tuning defenses.
  arXiv  Detail & Related papers  (2023-11-20T02:21:49Z)
• Backdoor Attack with Sparse and Invisible Trigger [57.41876708712008]
  Deep neural networks (DNNs) are vulnerable to backdoor attacks. backdoor attack is an emerging yet threatening training-phase threat. We propose a sparse and invisible backdoor attack (SIBA)
  arXiv  Detail & Related papers  (2023-05-11T10:05:57Z)
• Backdoor Defense via Suppressing Model Shortcuts [91.30995749139012]
  In this paper, we explore the backdoor mechanism from the angle of the model structure. We demonstrate that the attack success rate (ASR) decreases significantly when reducing the outputs of some key skip connections.
  arXiv  Detail & Related papers  (2022-11-02T15:39:19Z)
• Robust Deep Reinforcement Learning through Adversarial Loss [74.20501663956604]
  Recent studies have shown that deep reinforcement learning agents are vulnerable to small adversarial perturbations on the agent's inputs. We propose RADIAL-RL, a principled framework to train reinforcement learning agents with improved robustness against adversarial attacks.
  arXiv  Detail & Related papers  (2020-08-05T07:49:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.