Learning-Augmented Decentralized Online Convex Optimization in Networks

• URL: http://arxiv.org/abs/2306.10158v3
• Date: Fri, 18 Oct 2024 01:06:40 GMT
• Title: Learning-Augmented Decentralized Online Convex Optimization in Networks
• Authors: Pengfei Li, Jianyi Yang, Adam Wierman, Shaolei Ren,
• Abstract summary: This paper studies decentralized online convex optimization in a networked multi-agent system. It proposes a novel algorithm, Learning-Augmented Decentralized Online optimization (LADO) for individual agents to select actions only based on local online information.
• Score: 40.142341503145275
• License:
• Abstract: This paper studies decentralized online convex optimization in a networked multi-agent system and proposes a novel algorithm, Learning-Augmented Decentralized Online optimization (LADO), for individual agents to select actions only based on local online information. LADO leverages a baseline policy to safeguard online actions for worst-case robustness guarantees, while staying close to the machine learning (ML) policy for average performance improvement. In stark contrast with the existing learning-augmented online algorithms that focus on centralized settings, LADO achieves strong robustness guarantees in a decentralized setting. We also prove the average cost bound for LADO, revealing the tradeoff between average performance and worst-case robustness and demonstrating the advantage of training the ML policy by explicitly considering the robustness requirement.

Related papers

• RESIST: Resilient Decentralized Learning Using Consensus Gradient Descent [11.22833419439317]
  Empirical robustness risk (ERM) is a cornerstone of modern machine learning (ML) This paper focuses on the man-in-the-middle (MITM) attack, which can cause models to deviate significantly from their intended ERM solutions. We propose RESIST, an algorithm designed to be robust against adversarially compromised communication links.
  arXiv  Detail & Related papers  (2025-02-11T21:48:10Z)
• An Offline Multi-Agent Reinforcement Learning Framework for Radio Resource Management [5.771885923067511]
  offline multi-agent reinforcement learning (MARL) addresses key limitations of online MARL. We propose an offline MARL algorithm for radio resource management (RRM) We evaluate three training paradigms: centralized, independent, and centralized training with decentralized execution (CTDE)
  arXiv  Detail & Related papers  (2025-01-22T16:25:46Z)
• Self-Supervised Learning for Large-Scale Preventive Security Constrained DC Optimal Power Flow [20.078717680640214]
  Security-Constrained Optimal Power Flow (SCOPF) plays a crucial role in power grid stability but becomes increasingly complex as systems grow. This paper introduces PDL-SCOPF, a self-supervised end-to-end primal-dual learning framework for producing near-optimal solutions to large-scale SCOPF problems.
  arXiv  Detail & Related papers  (2023-11-29T20:36:35Z)
• Imitation Learning based Alternative Multi-Agent Proximal Policy Optimization for Well-Formed Swarm-Oriented Pursuit Avoidance [15.498559530889839]
  In this paper, we put forward a decentralized learning based Alternative Multi-Agent Proximal Policy Optimization (IA-MAPPO) algorithm to execute the pursuit avoidance task in well-formed swarm. We utilize imitation learning to decentralize the formation controller, so as to reduce the communication overheads and enhance the scalability. The simulation results validate the effectiveness of IA-MAPPO and extensive ablation experiments further show the performance comparable to a centralized solution with significant decrease in communication overheads.
  arXiv  Detail & Related papers  (2023-11-06T06:58:16Z)
• Decentralized Stochastic Optimization with Inherent Privacy Protection [103.62463469366557]
  Decentralized optimization is the basic building block of modern collaborative machine learning, distributed estimation and control, and large-scale sensing. Since involved data, privacy protection has become an increasingly pressing need in the implementation of decentralized optimization algorithms.
  arXiv  Detail & Related papers  (2022-05-08T14:38:23Z)
• Supported Policy Optimization for Offline Reinforcement Learning [74.1011309005488]
  Policy constraint methods to offline reinforcement learning (RL) typically utilize parameterization or regularization. Regularization methods reduce the divergence between the learned policy and the behavior policy. This paper presents Supported Policy OpTimization (SPOT), which is directly derived from the theoretical formalization of the density-based support constraint.
  arXiv  Detail & Related papers  (2022-02-13T07:38:36Z)
• Adaptive Stochastic ADMM for Decentralized Reinforcement Learning in Edge Industrial IoT [106.83952081124195]
  Reinforcement learning (RL) has been widely investigated and shown to be a promising solution for decision-making and optimal control processes. We propose an adaptive ADMM (asI-ADMM) algorithm and apply it to decentralized RL with edge-computing-empowered IIoT networks. Experiment results show that our proposed algorithms outperform the state of the art in terms of communication costs and scalability, and can well adapt to complex IoT environments.
  arXiv  Detail & Related papers  (2021-06-30T16:49:07Z)
• OptiDICE: Offline Policy Optimization via Stationary Distribution Correction Estimation [59.469401906712555]
  We present an offline reinforcement learning algorithm that prevents overestimation in a more principled way. Our algorithm, OptiDICE, directly estimates the stationary distribution corrections of the optimal policy. We show that OptiDICE performs competitively with the state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-06-21T00:43:30Z)
• Two-stage Deep Reinforcement Learning for Inverter-based Volt-VAR Control in Active Distribution Networks [3.260913246106564]
  We propose a novel two-stage deep reinforcement learning (DRL) method to improve the voltage profile by regulating inverter-based energy resources. In the offline stage, a highly efficient adversarial reinforcement learning algorithm is developed to train an offline agent robust to the model mismatch. In the sequential online stage, we transfer the offline agent safely as the online agent to perform continuous learning and controlling online with significantly improved safety and efficiency.
  arXiv  Detail & Related papers  (2020-05-20T08:02:13Z)
• Decentralized MCTS via Learned Teammate Models [89.24858306636816]
  We present a trainable online decentralized planning algorithm based on decentralized Monte Carlo Tree Search. We show that deep learning and convolutional neural networks can be employed to produce accurate policy approximators.
  arXiv  Detail & Related papers  (2020-03-19T13:10:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.