Using Reinforcement Learning to Herd a Robotic Swarm to a Target Distribution

• URL: http://arxiv.org/abs/2006.15807v2
• Date: Sat, 12 Dec 2020 20:52:26 GMT
• Title: Using Reinforcement Learning to Herd a Robotic Swarm to a Target Distribution
• Abstract summary: We present a reinforcement learning approach to designing a control policy for a "leader" agent that herds a swarm of "follower" agents. Two Temporal-Difference learning algorithms, SARSA and Q-Learning, are used to generate the leader control policy.
• Score: 3.706222947143855
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we present a reinforcement learning approach to designing a control policy for a "leader" agent that herds a swarm of "follower" agents, via repulsive interactions, as quickly as possible to a target probability distribution over a strongly connected graph. The leader control policy is a function of the swarm distribution, which evolves over time according to a mean-field model in the form of an ordinary difference equation. The dependence of the policy on agent populations at each graph vertex, rather than on individual agent activity, simplifies the observations required by the leader and enables the control strategy to scale with the number of agents. Two Temporal-Difference learning algorithms, SARSA and Q-Learning, are used to generate the leader control policy based on the follower agent distribution and the leader's location on the graph. A simulation environment corresponding to a grid graph with 4 vertices was used to train and validate the control policies for follower agent populations ranging from 10 to 100. Finally, the control policies trained on 100 simulated agents were used to successfully redistribute a physical swarm of 10 small robots to a target distribution among 4 spatial regions.

Related papers

• Planner-Conditioned Diffusion for Coordinated Multi-Agent Exploration [7.151235176774153]
  Coordinated multi-agent exploration requires efficient individual coverage and non-redundant coverage across agents.<n>We propose a Planner-Conditioned Diffusion Policy (PCDP) for graph-based multi-agent exploration.
  arXiv  Detail & Related papers  (2026-08-17T08:07:20Z)
• Self-evolving LLM agents with in-distribution Optimization [60.05867547965365]
  Large Language Models (LLMs) have recently emerged as powerful controllers for interactive agents in complex environments.<n>We propose Q-Evolve, a self-evolving framework for LLM agents that unifies automatic process-reward labeling and policy learning.<n>We evaluate our method on AlfWorld, WebShop, and ScienceWorld, showing Q-Evolve outperforms strong baselines in sample efficiency, robustness, and overall task performance.
  arXiv  Detail & Related papers  (2026-06-05T15:09:52Z)
• Decentralized Learning Strategies for Estimation Error Minimization with Graph Neural Networks [86.99017195607077]
  We address real-time sampling and estimation of autoregressive Markovian sources in wireless networks.<n>We propose a graphical reinforcement learning framework for policy optimization.<n>Theoretically, our proposed policies are transferable, allowing a policy trained on one graph to be effectively applied to structurally similar graphs.
  arXiv  Detail & Related papers  (2026-01-19T02:18:45Z)
• On Swarm Leader Identification using Probing Policies [2.707154152696381]
  This work introduces the interactive Swarm Leader Identification problem.<n>An adversarial probing agent identifies a swarm's leader by physically interacting with its members.<n>We employ Deep Reinforcement Learning to train the prober's policy.
  arXiv  Detail & Related papers  (2025-12-20T00:02:58Z)
• Bayesian Ego-graph inference for Networked Multi-Agent Reinforcement Learning [16.190458233440864]
  We propose a graph-based policy for Networked-MARL, where each agent conditions its decision on a sampled subgraph over its local physical neighborhood.<n>We introduce BayesG, a decentralized actor-framework that learns sparse, context-aware interaction structures via Bayesian variational inference.<n>BayesG outperforms strong MARL baselines on large-scale traffic control tasks with up to 167 agents.
  arXiv  Detail & Related papers  (2025-09-20T10:09:37Z)
• Power Grid Control with Graph-Based Distributed Reinforcement Learning [60.49805771047161]
  This work advances a graph-based distributed reinforcement learning framework for real-time, scalable grid management.<n>A Graph Neural Network (GNN) is employed to encode the network's topological information within the single low-level agent's observation.<n>Experiments on the Grid2Op simulation environment show the effectiveness of the approach.
  arXiv  Detail & Related papers  (2025-09-02T22:17:25Z)
• Make-An-Agent: A Generalizable Policy Network Generator with Behavior-Prompted Diffusion [41.52811286996212]
  Make-An-Agent is a novel policy parameter generator for behavior-to-policy generation. We show how it can generate a control policy for an agent using just one demonstration of desired behaviors as a prompt. We also deploy policies generated by Make-An-Agent onto real-world robots on locomotion tasks.
  arXiv  Detail & Related papers  (2024-07-15T17:59:57Z)
• Decentralized Learning Strategies for Estimation Error Minimization with Graph Neural Networks [94.2860766709971]
  We address the challenge of sampling and remote estimation for autoregressive Markovian processes in a wireless network with statistically-identical agents. Our goal is to minimize time-average estimation error and/or age of information with decentralized scalable sampling and transmission policies.
  arXiv  Detail & Related papers  (2024-04-04T06:24:11Z)
• Distributed Policy Gradient for Linear Quadratic Networked Control with Limited Communication Range [23.500806437272487]
  We show that it is possible to approximate the exact gradient only using local information. Compared with the centralized optimal controller, the performance gap decreases to zero exponentially as the communication and control ranges increase.
  arXiv  Detail & Related papers  (2024-03-05T15:38:54Z)
• Combat Urban Congestion via Collaboration: Heterogeneous GNN-based MARL for Coordinated Platooning and Traffic Signal Control [14.302023260372563]
  This paper proposes an innovative solution to tackle challenges based on heterogeneous graph multi-agent reinforcement learning and traffic theories. We design platoon and signal control as distinct reinforcement learning agents with their own set of observations, actions, and reward functions to optimize traffic flow. We evaluate our approach through SUMO simulations, which show convergent results in terms of both travel time and fuel consumption, and superior performance compared to other adaptive signal control methods.
  arXiv  Detail & Related papers  (2023-10-17T02:46:04Z)
• A Policy Iteration Approach for Flock Motion Control [5.419608513284392]
  The overall control process guides the agents while monitoring the flock-cohesiveness and localization. An online model-free policy iteration mechanism is developed here to guide a flock of agents to follow an independent command generator. The simulation results of the policy iteration mechanism revealed fast learning and convergence behaviors with less computational effort.
  arXiv  Detail & Related papers  (2023-03-17T15:04:57Z)
• Distributed-Training-and-Execution Multi-Agent Reinforcement Learning for Power Control in HetNet [48.96004919910818]
  We propose a multi-agent deep reinforcement learning (MADRL) based power control scheme for the HetNet. To promote cooperation among agents, we develop a penalty-based Q learning (PQL) algorithm for MADRL systems. In this way, an agent's policy can be learned by other agents more easily, resulting in a more efficient collaboration process.
  arXiv  Detail & Related papers  (2022-12-15T17:01:56Z)
• Explaining Reinforcement Learning Policies through Counterfactual Trajectories [147.7246109100945]
  A human developer must validate that an RL agent will perform well at test-time. Our method conveys how the agent performs under distribution shifts by showing the agent's behavior across a wider trajectory distribution. In a user study, we demonstrate that our method enables users to score better than baseline methods on one of two agent validation tasks.
  arXiv  Detail & Related papers  (2022-01-29T00:52:37Z)
• Learning Connectivity for Data Distribution in Robot Teams [96.39864514115136]
  We propose a task-agnostic, decentralized, low-latency method for data distribution in ad-hoc networks using Graph Neural Networks (GNN) Our approach enables multi-agent algorithms based on global state information to function by ensuring it is available at each robot. We train the distributed GNN communication policies via reinforcement learning using the average Age of Information as the reward function and show that it improves training stability compared to task-specific reward functions.
  arXiv  Detail & Related papers  (2021-03-08T21:48:55Z)
• Implicit Distributional Reinforcement Learning [61.166030238490634]
  implicit distributional actor-critic (IDAC) built on two deep generator networks (DGNs) Semi-implicit actor (SIA) powered by a flexible policy distribution. We observe IDAC outperforms state-of-the-art algorithms on representative OpenAI Gym environments.
  arXiv  Detail & Related papers  (2020-07-13T02:52:18Z)
• Distributed Voltage Regulation of Active Distribution System Based on Enhanced Multi-agent Deep Reinforcement Learning [9.7314654861242]
  This paper proposes a data-driven distributed voltage control approach based on the spectrum clustering and the enhanced multi-agent deep reinforcement learning (MADRL) algorithm. The proposed method can significantly reduce the requirements of communications and knowledge of system parameters. It also effectively deals with uncertainties and can provide online coordinated control based on the latest local information.
  arXiv  Detail & Related papers  (2020-05-31T15:48:27Z)
• Model-based Reinforcement Learning for Decentralized Multiagent Rendezvous [66.6895109554163]
  Underlying the human ability to align goals with other agents is their ability to predict the intentions of others and actively update their own plans. We propose hierarchical predictive planning (HPP), a model-based reinforcement learning method for decentralized multiagent rendezvous.
  arXiv  Detail & Related papers  (2020-03-15T19:49:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.