Decentralized Multi-Robot Formation Control Using Reinforcement Learning

• URL: http://arxiv.org/abs/2306.14489v1
• Date: Mon, 26 Jun 2023 08:02:55 GMT
• Title: Decentralized Multi-Robot Formation Control Using Reinforcement Learning
• Authors: Juraj Obradovic, Marko Krizmancic, Stjepan Bogdan
• Abstract summary: This paper presents a decentralized leader-follower multi-robot formation control based on a reinforcement learning (RL) algorithm applied to a swarm of small educational Sphero robots. To enhance the system behavior, we trained two different DDQN models, one for reaching the formation and the other for maintaining it. The presented approach has been tested in simulation and real experiments which show that the multi-robot system can achieve and maintain a stable formation without the need for complex mathematical models and nonlinear control laws.
• Score: 2.7716102039510564
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: This paper presents a decentralized leader-follower multi-robot formation control based on a reinforcement learning (RL) algorithm applied to a swarm of small educational Sphero robots. Since the basic Q-learning method is known to require large memory resources for Q-tables, this work implements the Double Deep Q-Network (DDQN) algorithm, which has achieved excellent results in many robotic problems. To enhance the system behavior, we trained two different DDQN models, one for reaching the formation and the other for maintaining it. The models use a discrete set of robot motions (actions) to adapt the continuous nonlinear system to the discrete nature of RL. The presented approach has been tested in simulation and real experiments which show that the multi-robot system can achieve and maintain a stable formation without the need for complex mathematical models and nonlinear control laws.

Related papers

• Modelling, Positioning, and Deep Reinforcement Learning Path Tracking Control of Scaled Robotic Vehicles: Design and Experimental Validation [3.807917169053206]
  Scaled robotic cars are commonly equipped with a hierarchical control acthiecture that includes tasks dedicated to vehicle state estimation and control. This paper covers both aspects by proposing (i) a federeted extended Kalman filter (FEKF) and (ii) a novel deep reinforcement learning (DRL) path tracking controller trained via an expert demonstrator. The experimentally validated model is used for (i) supporting the design of the FEKF and (ii) serving as a digital twin for training the proposed DRL-based path tracking algorithm.
  arXiv  Detail & Related papers  (2024-01-10T14:40:53Z)
• Robot Fine-Tuning Made Easy: Pre-Training Rewards and Policies for Autonomous Real-World Reinforcement Learning [58.3994826169858]
  We introduce RoboFuME, a reset-free fine-tuning system for robotic reinforcement learning. Our insights are to utilize offline reinforcement learning techniques to ensure efficient online fine-tuning of a pre-trained policy. Our method can incorporate data from an existing robot dataset and improve on a target task within as little as 3 hours of autonomous real-world experience.
  arXiv  Detail & Related papers  (2023-10-23T17:50:08Z)
• Stabilizing Contrastive RL: Techniques for Robotic Goal Reaching from Offline Data [101.43350024175157]
  Self-supervised learning has the potential to decrease the amount of human annotation and engineering effort required to learn control strategies. Our work builds on prior work showing that the reinforcement learning (RL) itself can be cast as a self-supervised problem. We demonstrate that a self-supervised RL algorithm based on contrastive learning can solve real-world, image-based robotic manipulation tasks.
  arXiv  Detail & Related papers  (2023-06-06T01:36:56Z)
• Real-to-Sim: Predicting Residual Errors of Robotic Systems with Sparse Data using a Learning-based Unscented Kalman Filter [65.93205328894608]
  We learn the residual errors between a dynamic and/or simulator model and the real robot. We show that with the learned residual errors, we can further close the reality gap between dynamic models, simulations, and actual hardware.
  arXiv  Detail & Related papers  (2022-09-07T15:15:12Z)
• Bridging Model-based Safety and Model-free Reinforcement Learning through System Identification of Low Dimensional Linear Models [16.511440197186918]
  We propose a new method to combine model-based safety with model-free reinforcement learning. We show that a low-dimensional dynamical model is sufficient to capture the dynamics of the closed-loop system. We illustrate that the found linear model is able to provide guarantees by safety-critical optimal control framework.
  arXiv  Detail & Related papers  (2022-05-11T22:03:18Z)
• Training and Evaluation of Deep Policies using Reinforcement Learning and Generative Models [67.78935378952146]
  GenRL is a framework for solving sequential decision-making problems. It exploits the combination of reinforcement learning and latent variable generative models. We experimentally determine the characteristics of generative models that have most influence on the performance of the final policy training.
  arXiv  Detail & Related papers  (2022-04-18T22:02:32Z)
• Learning to Walk Autonomously via Reset-Free Quality-Diversity [73.08073762433376]
  Quality-Diversity algorithms can discover large and complex behavioural repertoires consisting of both diverse and high-performing skills. Existing QD algorithms need large numbers of evaluations as well as episodic resets, which require manual human supervision and interventions. This paper proposes Reset-Free Quality-Diversity optimization (RF-QD) as a step towards autonomous learning for robotics in open-ended environments.
  arXiv  Detail & Related papers  (2022-04-07T14:07:51Z)
• Decentralized Circle Formation Control for Fish-like Robots in the Real-world via Reinforcement Learning [16.050280167846264]
  Circle formation control problem is addressed for a group of cooperative underactuated fish-like robots. We propose a decentralized controller without the knowledge of the dynamics of the fish-like robots.
  arXiv  Detail & Related papers  (2021-03-09T08:38:28Z)
• Model-Free Control of Dynamical Systems with Deep Reservoir Computing [0.0]
  We propose and demonstrate a nonlinear control method that can be applied to unknown, complex systems. Our technique requires no prior knowledge of the system and is thus model-free. Reservoir computers are well-suited to the control problem because they require small training data sets and remarkably low training times.
  arXiv  Detail & Related papers  (2020-10-05T18:59:51Z)
• Learning to Walk: Spike Based Reinforcement Learning for Hexapod Robot Central Pattern Generation [2.4603302139672003]
  Methods such as gradient, deep reinforcement learning (RL) have been explored for bipeds, quadrupeds and hexapods. Recent advances in spiking neural networks (SNNs) promise a significant reduction in computing owing to the sparse firing of neuros. We propose a reinforcement based weight update technique for training a spiking pattern generator.
  arXiv  Detail & Related papers  (2020-03-22T23:45:32Z)
• Information Theoretic Model Predictive Q-Learning [64.74041985237105]
  We present a novel theoretical connection between information theoretic MPC and entropy regularized RL. We develop a Q-learning algorithm that can leverage biased models.
  arXiv  Detail & Related papers  (2019-12-31T00:29:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.