Cooperative Path Integral Control for Stochastic Multi-Agent Systems

• URL: http://arxiv.org/abs/2009.14775v2
• Date: Sun, 21 Mar 2021 03:28:03 GMT
• Title: Cooperative Path Integral Control for Stochastic Multi-Agent Systems
• Authors: Neng Wan, Aditya Gahlawat, Naira Hovakimyan, Evangelos A. Theodorou, and Petros G. Voulgaris
• Abstract summary: A distributed optimal control solution is presented for cooperative multi-agent systems. Local control actions that rely only on agents' local observations are designed to optimize the joint cost functions of subsystems.
• Score: 20.731989147508983
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A distributed stochastic optimal control solution is presented for cooperative multi-agent systems. The network of agents is partitioned into multiple factorial subsystems, each of which consists of a central agent and neighboring agents. Local control actions that rely only on agents' local observations are designed to optimize the joint cost functions of subsystems. When solving for the local control actions, the joint optimality equation for each subsystem is cast as a linear partial differential equation and solved using the Feynman-Kac formula. The solution and the optimal control action are then formulated as path integrals and approximated by a Monte-Carlo method. Numerical verification is provided through a simulation example consisting of a team of cooperative UAVs.

Related papers

• Hypernetwork-based approach for optimal composition design in partially controlled multi-agent systems [5.860363407227059]
  Partially Controlled Multi-Agent Systems (PCMAS) are comprised of controllable agents, managed by a system designer, and uncontrollable agents, operating autonomously. This study addresses an optimal composition design problem in PCMAS, which involves the system designer's problem, determining the optimal number and policies of controllable agents, and the uncontrollable agents' problem. We propose a novel hypernetwork-based framework that jointly optimize the system's composition and agent policies.
  arXiv  Detail & Related papers  (2025-02-18T07:35:24Z)
• Cluster-Based Multi-Agent Task Scheduling for Space-Air-Ground Integrated Networks [60.085771314013044]
  Low-altitude economy holds significant potential for development in areas such as communication and sensing. We propose a Clustering-based Multi-agent Deep Deterministic Policy Gradient (CMADDPG) algorithm to address the multi-UAV cooperative task scheduling challenges in SAGIN.
  arXiv  Detail & Related papers  (2024-12-14T06:17:33Z)
• Stochastic Optimal Control Matching [53.156277491861985]
  Our work introduces Optimal Control Matching (SOCM), a novel Iterative Diffusion Optimization (IDO) technique for optimal control. The control is learned via a least squares problem by trying to fit a matching vector field. Experimentally, our algorithm achieves lower error than all the existing IDO techniques for optimal control.
  arXiv  Detail & Related papers  (2023-12-04T16:49:43Z)
• Optimal State Manipulation for a Two-Qubit System Driven by Coherent and Incoherent Controls [77.34726150561087]
  State preparation is important for optimal control of two-qubit quantum systems. We exploit two physically different coherent control and optimize the Hilbert-Schmidt target density matrices.
  arXiv  Detail & Related papers  (2023-04-03T10:22:35Z)
• Fully Decentralized, Scalable Gaussian Processes for Multi-Agent Federated Learning [14.353574903736343]
  We propose decentralized and scalable algorithms for GP training and prediction in multi-agent systems. The efficacy of the proposed methods is illustrated with numerical experiments on synthetic and real data.
  arXiv  Detail & Related papers  (2022-03-06T02:54:13Z)
• Multi-Agent MDP Homomorphic Networks [100.74260120972863]
  In cooperative multi-agent systems, complex symmetries arise between different configurations of the agents and their local observations. Existing work on symmetries in single agent reinforcement learning can only be generalized to the fully centralized setting. This paper introduces Multi-Agent MDP Homomorphic Networks, a class of networks that allows distributed execution using only local information.
  arXiv  Detail & Related papers  (2021-10-09T07:46:25Z)
• Efficient Model-Based Multi-Agent Mean-Field Reinforcement Learning [89.31889875864599]
  We propose an efficient model-based reinforcement learning algorithm for learning in multi-agent systems. Our main theoretical contributions are the first general regret bounds for model-based reinforcement learning for MFC. We provide a practical parametrization of the core optimization problem.
  arXiv  Detail & Related papers  (2021-07-08T18:01:02Z)
• Distributed Algorithms for Linearly-Solvable Optimal Control in Networked Multi-Agent Systems [15.782670973813774]
  A distributed framework is proposed to partition the optimal control problem of a networked MAS into several local optimal control problems. For discrete-time systems, the joint Bellman equation of each subsystem is transformed into a system of linear equations. For continuous-time systems, the joint optimality equation of each subsystem is converted into a linear partial differential equation.
  arXiv  Detail & Related papers  (2021-02-18T01:31:17Z)
• Compositionality of Linearly Solvable Optimal Control in Networked Multi-Agent Systems [27.544923751902807]
  We discuss the methodology of generalizing the optimal control law from learned component tasks to unlearned composite tasks on Multi-Agent Systems (MASs) The proposed approach achieves both the compositionality and optimality of control actions simultaneously within the cooperative MAS framework in both discrete- and continuous-time in a sample-efficient manner.
  arXiv  Detail & Related papers  (2020-09-28T20:21:48Z)
• A Multi-Agent Primal-Dual Strategy for Composite Optimization over Distributed Features [52.856801164425086]
  We study multi-agent sharing optimization problems with the objective function being the sum of smooth local functions plus a convex (possibly non-smooth) coupling function.
  arXiv  Detail & Related papers  (2020-06-15T19:40:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.