An Energy-aware and Fault-tolerant Deep Reinforcement Learning based approach for Multi-agent Patrolling Problems

• URL: http://arxiv.org/abs/2212.08230v4
• Date: Fri, 9 Jun 2023 03:22:52 GMT
• Title: An Energy-aware and Fault-tolerant Deep Reinforcement Learning based approach for Multi-agent Patrolling Problems
• Authors: Chenhao Tong, Aaron Harwood, Maria A. Rodriguez, Richard O. Sinnott
• Abstract summary: We propose an approach based on model-free, deep multi-agent reinforcement learning. Agents are trained to patrol an environment with various unknown dynamics and factors. They can automatically recharge themselves to support continuous collective patrolling. This architecture provides a patrolling system that can tolerate agent failures and allow supplementary agents to be added to replace failed agents or to increase the overall patrol performance.
• Score: 0.5008597638379226
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Autonomous vehicles are suited for continuous area patrolling problems. However, finding an optimal patrolling strategy can be challenging for many reasons. Firstly, patrolling environments are often complex and can include unknown environmental factors, such as wind or landscape. Secondly, autonomous vehicles can have failures or hardware constraints, such as limited battery life. Importantly, patrolling large areas often requires multiple agents that need to collectively coordinate their actions. In this work, we consider these limitations and propose an approach based on model-free, deep multi-agent reinforcement learning. In this approach, the agents are trained to patrol an environment with various unknown dynamics and factors. They can automatically recharge themselves to support continuous collective patrolling. A distributed homogeneous multi-agent architecture is proposed, where all patrolling agents execute identical policies locally based on their local observations and shared location information. This architecture provides a patrolling system that can tolerate agent failures and allow supplementary agents to be added to replace failed agents or to increase the overall patrol performance. The solution is validated through simulation experiments from multiple perspectives, including the overall patrol performance, the efficiency of battery recharging strategies, the overall fault tolerance, and the ability to cooperate with supplementary agents.

Related papers

• Improving the Generalization of Unseen Crowd Behaviors for Reinforcement Learning based Local Motion Planners [36.684452789236914]
  Current Reinforcement Learning-based motion planners rely on a single policy to simulate pedestrian movements. We introduce an efficient method that enhances agent diversity within a single policy by maximizing an information-theoretic objective. In assessing an agent's robustness against unseen crowds, we propose diverse scenarios inspired by pedestrian crowd behaviors.
  arXiv  Detail & Related papers  (2024-10-16T04:46:21Z)
• Multi-Agent Reinforcement Learning for Joint Police Patrol and Dispatch [13.336551874123796]
  We propose a novel method for jointly optimizing multi-agent patrol and dispatch to learn policies yielding rapid response times. Our method treats each patroller as an independent Q-learner (agent) with a shared deep Q-network that represents the state-action values. We demonstrate that this heterogeneous multi-agent reinforcement learning approach is capable of learning policies that optimize for patrol or dispatch alone.
  arXiv  Detail & Related papers  (2024-09-03T19:19:57Z)
• KoMA: Knowledge-driven Multi-agent Framework for Autonomous Driving with Large Language Models [15.951550445568605]
  Large language models (LLMs) as autonomous agents offer a novel avenue for tackling real-world challenges through a knowledge-driven manner. We propose the KoMA framework consisting of multi-agent interaction, multi-step planning, shared-memory, and ranking-based reflection modules.
  arXiv  Detail & Related papers  (2024-07-19T12:13:08Z)
• Autonomous Vehicle Patrolling Through Deep Reinforcement Learning: Learning to Communicate and Cooperate [3.79830302036482]
  Finding an optimal patrolling strategy can be challenging due to unknown environmental factors, such as wind or landscape. Agents are trained to develop their own communication protocol to cooperate during patrolling where faults can and do occur. The solution is validated through simulation experiments and is compared with several state-of-the-art patrolling solutions from different perspectives.
  arXiv  Detail & Related papers  (2024-01-28T14:29:30Z)
• Safe Model-Based Multi-Agent Mean-Field Reinforcement Learning [48.667697255912614]
  Mean-field reinforcement learning addresses the policy of a representative agent interacting with the infinite population of identical agents. We propose Safe-M$3$-UCRL, the first model-based mean-field reinforcement learning algorithm that attains safe policies even in the case of unknown transitions. Our algorithm effectively meets the demand in critical areas while ensuring service accessibility in regions with low demand.
  arXiv  Detail & Related papers  (2023-06-29T15:57:07Z)
• Guaranteed Discovery of Controllable Latent States with Multi-Step Inverse Models [51.754160866582005]
  Agent-Controllable State Discovery algorithm (AC-State) Algorithm consists of a multi-step inverse model (predicting actions from distant observations) with an information bottleneck. We demonstrate the discovery of controllable latent state in three domains: localizing a robot arm with distractions, exploring in a maze alongside other agents, and navigating in the Matterport house simulator.
  arXiv  Detail & Related papers  (2022-07-17T17:06:52Z)
• Coach-assisted Multi-Agent Reinforcement Learning Framework for Unexpected Crashed Agents [120.91291581594773]
  We present a formal formulation of a cooperative multi-agent reinforcement learning system with unexpected crashes. We propose a coach-assisted multi-agent reinforcement learning framework, which introduces a virtual coach agent to adjust the crash rate during training. To the best of our knowledge, this work is the first to study the unexpected crashes in the multi-agent system.
  arXiv  Detail & Related papers  (2022-03-16T08:22:45Z)
• Cooperative Exploration for Multi-Agent Deep Reinforcement Learning [127.4746863307944]
  We propose cooperative multi-agent exploration (CMAE) for deep reinforcement learning. The goal is selected from multiple projected state spaces via a normalized entropy-based technique. We demonstrate that CMAE consistently outperforms baselines on various tasks.
  arXiv  Detail & Related papers  (2021-07-23T20:06:32Z)
• Explore and Control with Adversarial Surprise [78.41972292110967]
  Reinforcement learning (RL) provides a framework for learning goal-directed policies given user-specified rewards. We propose a new unsupervised RL technique based on an adversarial game which pits two policies against each other to compete over the amount of surprise an RL agent experiences. We show that our method leads to the emergence of complex skills by exhibiting clear phase transitions.
  arXiv  Detail & Related papers  (2021-07-12T17:58:40Z)
• Heterogeneous Multi-Agent Reinforcement Learning for Unknown Environment Mapping [0.0]
  We present an actor-critic algorithm that allows a team of heterogeneous agents to learn decentralized control policies for covering an unknown environment. This task is of interest to national security and emergency response organizations that would like to enhance situational awareness in hazardous areas by deploying teams of unmanned aerial vehicles.
  arXiv  Detail & Related papers  (2020-10-06T12:23:05Z)
• ReLMoGen: Leveraging Motion Generation in Reinforcement Learning for Mobile Manipulation [99.2543521972137]
  ReLMoGen is a framework that combines a learned policy to predict subgoals and a motion generator to plan and execute the motion needed to reach these subgoals. Our method is benchmarked on a diverse set of seven robotics tasks in photo-realistic simulation environments. ReLMoGen shows outstanding transferability between different motion generators at test time, indicating a great potential to transfer to real robots.
  arXiv  Detail & Related papers  (2020-08-18T08:05:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.