ARBoids: Adaptive Residual Reinforcement Learning With Boids Model for Cooperative Multi-USV Target Defense

• URL: http://arxiv.org/abs/2502.18549v2
• Date: Thu, 10 Jul 2025 05:08:34 GMT
• Title: ARBoids: Adaptive Residual Reinforcement Learning With Boids Model for Cooperative Multi-USV Target Defense
• Authors: Jiyue Tao, Tongsheng Shen, Dexin Zhao, Feitian Zhang,
• Abstract summary: This letter introduces ARBoids, a novel adaptive residual reinforcement learning framework.<n>It integrates deep reinforcement learning with the biologically inspired, force-based Boids model.<n>The proposed approach is validated in a high-fidelity Gazebo simulation environment.
• Score: 0.918715978278858
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The target defense problem (TDP) for unmanned surface vehicles (USVs) concerns intercepting an adversarial USV before it breaches a designated target region, using one or more defending USVs. A particularly challenging scenario arises when the attacker exhibits superior maneuverability compared to the defenders, significantly complicating effective interception. To tackle this challenge, this letter introduces ARBoids, a novel adaptive residual reinforcement learning framework that integrates deep reinforcement learning (DRL) with the biologically inspired, force-based Boids model. Within this framework, the Boids model serves as a computationally efficient baseline policy for multi-agent coordination, while DRL learns a residual policy to adaptively refine and optimize the defenders' actions. The proposed approach is validated in a high-fidelity Gazebo simulation environment, demonstrating superior performance over traditional interception strategies, including pure force-based approaches and vanilla DRL policies. Furthermore, the learned policy exhibits strong adaptability to attackers with diverse maneuverability profiles, highlighting its robustness and generalization capability. The code of ARBoids will be released upon acceptance of this letter.

Related papers

• Reinforcement Learning for Decision-Level Interception Prioritization in Drone Swarm Defense [56.47577824219207]
  We present a case study demonstrating the practical advantages of reinforcement learning in addressing this challenge.<n>We introduce a high-fidelity simulation environment that captures realistic operational constraints.<n>Agent learns to coordinate multiple effectors for optimal interception prioritization.<n>We evaluate the learned policy against a handcrafted rule-based baseline across hundreds of simulated attack scenarios.
  arXiv  Detail & Related papers  (2025-08-01T13:55:39Z)
• Robust Policy Switching for Antifragile Reinforcement Learning for UAV Deconfliction in Adversarial Environments [6.956559003734227]
  An unmanned aerial vehicles (UAVs) has been exposed to adversarial attacks that exploit vulnerabilities in reinforcement learning (RL)<n>This paper introduces an antifragile RL framework that enhances adaptability to broader distributional shifts.<n>It achieves superior performance, demonstrating shorter navigation path lengths and a higher rate of conflict-free navigation trajectories.
  arXiv  Detail & Related papers  (2025-06-26T10:06:29Z)
• Improving Large Language Model Safety with Contrastive Representation Learning [92.79965952162298]
  Large Language Models (LLMs) are powerful tools with profound societal impacts.<n>Their ability to generate responses to diverse and uncontrolled inputs leaves them vulnerable to adversarial attacks.<n>We propose a defense framework that formulates model defense as a contrastive representation learning problem.
  arXiv  Detail & Related papers  (2025-06-13T16:42:09Z)
• MISLEADER: Defending against Model Extraction with Ensembles of Distilled Models [56.09354775405601]
  Model extraction attacks aim to replicate the functionality of a black-box model through query access.<n>Most existing defenses presume that attacker queries have out-of-distribution (OOD) samples, enabling them to detect and disrupt suspicious inputs.<n>We propose MISLEADER, a novel defense strategy that does not rely on OOD assumptions.
  arXiv  Detail & Related papers  (2025-06-03T01:37:09Z)
• Optimal Actuator Attacks on Autonomous Vehicles Using Reinforcement Learning [11.836584342902492]
  We propose a reinforcement learning (RL)-based approach for designing optimal stealthy integrity attacks on AV actuators.<n>We also analyze the limitations of state-of-the-art RL-based secure controllers to counter such attacks.
  arXiv  Detail & Related papers  (2025-02-11T03:01:05Z)
• Adapting to Evolving Adversaries with Regularized Continual Robust Training [47.93633573641843]
  We present theoretical results which show that the gap in a model's robustness against different attacks is bounded by how far each attack perturbs a sample in the model's logit space. Our findings and open-source code lay the groundwork for the deployment of models robust to evolving attacks.
  arXiv  Detail & Related papers  (2025-02-06T17:38:41Z)
• Less is More: A Stealthy and Efficient Adversarial Attack Method for DRL-based Autonomous Driving Policies [2.9965913883475137]
  We present a stealthy and efficient adversarial attack method for DRL-based autonomous driving policies.<n>We train the adversary to learn the optimal policy for attacking at critical moments without domain knowledge.<n>Our method achieves more than 90% collision rate within three attacks in most cases.
  arXiv  Detail & Related papers  (2024-12-04T06:11:09Z)
• Sustainable Self-evolution Adversarial Training [51.25767996364584]
  We propose a Sustainable Self-Evolution Adversarial Training (SSEAT) framework for adversarial training defense models.<n>We introduce a continual adversarial defense pipeline to realize learning from various kinds of adversarial examples.<n>We also propose an adversarial data replay module to better select more diverse and key relearning data.
  arXiv  Detail & Related papers  (2024-12-03T08:41:11Z)
• Online Planning for Multi-UAV Pursuit-Evasion in Unknown Environments Using Deep Reinforcement Learning [16.676389371667284]
  Multi-UAV pursuit-evasion poses a key challenge for UAV swarm intelligence.<n>We introduce an evader prediction-enhanced network to tackle partial observability in cooperative strategy learning.<n>We derive a feasible policy via a two-stage reward refinement and deploy the policy on real quadrotors in a zero-shot manner.
  arXiv  Detail & Related papers  (2024-09-24T08:40:04Z)
• Towards Robust Policy: Enhancing Offline Reinforcement Learning with Adversarial Attacks and Defenses [19.918548094276005]
  offline reinforcement learning (RL) addresses the challenge of expensive and high-risk data exploration inherent in RL by pre-training policies on vast amounts of offline data. This paper proposes a framework to enhance the robustness of offline RL models by leveraging advanced adversarial attacks and defenses.
  arXiv  Detail & Related papers  (2024-05-18T07:23:44Z)
• Meta Invariance Defense Towards Generalizable Robustness to Unknown Adversarial Attacks [62.036798488144306]
  Current defense mainly focuses on the known attacks, but the adversarial robustness to the unknown attacks is seriously overlooked. We propose an attack-agnostic defense method named Meta Invariance Defense (MID) We show that MID simultaneously achieves robustness to the imperceptible adversarial perturbations in high-level image classification and attack-suppression in low-level robust image regeneration.
  arXiv  Detail & Related papers  (2024-04-04T10:10:38Z)
• The Adaptive Arms Race: Redefining Robustness in AI Security [21.759075171536388]
  We introduce a framework for adaptively optimizing black-box attacks and defenses under the competitive game they form.<n>We find that active defenses, those that dynamically control system responses, are an essential complement to model hardening against decision-based attacks.<n>Our findings, supported by an extensive theoretical and empirical investigation, confirm that adaptive adversaries pose a serious threat to black-box AI-based systems.
  arXiv  Detail & Related papers  (2023-12-20T21:24:52Z)
• Learn from the Past: A Proxy Guided Adversarial Defense Framework with Self Distillation Regularization [53.04697800214848]
  Adversarial Training (AT) is pivotal in fortifying the robustness of deep learning models. AT methods, relying on direct iterative updates for target model's defense, frequently encounter obstacles such as unstable training and catastrophic overfitting. We present a general proxy guided defense framework, LAST' (bf Learn from the Pbf ast)
  arXiv  Detail & Related papers  (2023-10-19T13:13:41Z)
• Downlink Power Allocation in Massive MIMO via Deep Learning: Adversarial Attacks and Training [62.77129284830945]
  This paper considers a regression problem in a wireless setting and shows that adversarial attacks can break the DL-based approach. We also analyze the effectiveness of adversarial training as a defensive technique in adversarial settings and show that the robustness of DL-based wireless system against attacks improves significantly.
  arXiv  Detail & Related papers  (2022-06-14T04:55:11Z)
• Attacking and Defending Deep Reinforcement Learning Policies [3.6985039575807246]
  We study robustness of DRL policies to adversarial attacks from the perspective of robust optimization. We propose a greedy attack algorithm, which tries to minimize the expected return of the policy without interacting with the environment, and a defense algorithm, which performs adversarial training in a max-min form.
  arXiv  Detail & Related papers  (2022-05-16T12:47:54Z)
• LAS-AT: Adversarial Training with Learnable Attack Strategy [82.88724890186094]
  "Learnable attack strategy", dubbed LAS-AT, learns to automatically produce attack strategies to improve the model robustness. Our framework is composed of a target network that uses AEs for training to improve robustness and a strategy network that produces attack strategies to control the AE generation.
  arXiv  Detail & Related papers  (2022-03-13T10:21:26Z)
• Guided Adversarial Attack for Evaluating and Enhancing Adversarial Defenses [59.58128343334556]
  We introduce a relaxation term to the standard loss, that finds more suitable gradient-directions, increases attack efficacy and leads to more efficient adversarial training. We propose Guided Adversarial Margin Attack (GAMA), which utilizes function mapping of the clean image to guide the generation of adversaries. We also propose Guided Adversarial Training (GAT), which achieves state-of-the-art performance amongst single-step defenses.
  arXiv  Detail & Related papers  (2020-11-30T16:39:39Z)
• Query-based Targeted Action-Space Adversarial Policies on Deep Reinforcement Learning Agents [23.580682320064714]
  This work investigates targeted attacks in the action-space domain, also commonly known as actuation attacks in CPS literature. We show that a query-based black-box attack model that generates optimal perturbations with respect to an adversarial goal can be formulated as another reinforcement learning problem. Experimental results showed that adversarial policies that only observe the nominal policy's output generate stronger attacks than adversarial policies that observe the nominal policy's input and output.
  arXiv  Detail & Related papers  (2020-11-13T20:25:48Z)
• Robust Reinforcement Learning using Adversarial Populations [118.73193330231163]
  Reinforcement Learning (RL) is an effective tool for controller design but can struggle with issues of robustness. We show that using a single adversary does not consistently yield robustness to dynamics variations under standard parametrizations of the adversary. We propose a population-based augmentation to the Robust RL formulation in which we randomly initialize a population of adversaries and sample from the population uniformly during training.
  arXiv  Detail & Related papers  (2020-08-04T20:57:32Z)
• Adversarial jamming attacks and defense strategies via adaptive deep reinforcement learning [12.11027948206573]
  In this paper, we consider a victim user that performs DRL-based dynamic channel access, and an attacker that executes DRLbased jamming attacks to disrupt the victim. Both the victim and attacker are DRL agents and can interact with each other, retrain their models, and adapt to opponents' policies. We propose three defense strategies to maximize the attacked victim's accuracy and evaluate their performances.
  arXiv  Detail & Related papers  (2020-07-12T18:16:00Z)
• Boosting Adversarial Training with Hypersphere Embedding [53.75693100495097]
  Adversarial training is one of the most effective defenses against adversarial attacks for deep learning models. In this work, we advocate incorporating the hypersphere embedding mechanism into the AT procedure. We validate our methods under a wide range of adversarial attacks on the CIFAR-10 and ImageNet datasets.
  arXiv  Detail & Related papers  (2020-02-20T08:42:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.