Conjectural Online Learning with First-order Beliefs in Asymmetric Information Stochastic Games

• URL: http://arxiv.org/abs/2402.18781v4
• Date: Mon, 19 Aug 2024 16:45:42 GMT
• Title: Conjectural Online Learning with First-order Beliefs in Asymmetric Information Stochastic Games
• Authors: Tao Li, Kim Hammar, Rolf Stadler, Quanyan Zhu,
• Abstract summary: Asymmetric information games (AISGs) arise in many complex socio-technical systems. We propose conjectural online learning (COL), an online learning method under generic information structures in AISGs.
• Score: 13.33996350474556
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Asymmetric information stochastic games (AISGs) arise in many complex socio-technical systems, such as cyber-physical systems and IT infrastructures. Existing computational methods for AISGs are primarily offline and can not adapt to equilibrium deviations. Further, current methods are limited to particular information structures to avoid belief hierarchies. Considering these limitations, we propose conjectural online learning (COL), an online learning method under generic information structures in AISGs. COL uses a forecaster-actor-critic (FAC) architecture, where subjective forecasts are used to conjecture the opponents' strategies within a lookahead horizon, and Bayesian learning is used to calibrate the conjectures. To adapt strategies to nonstationary environments based on information feedback, COL uses online rollout with cost function approximation (actor-critic). We prove that the conjectures produced by COL are asymptotically consistent with the information feedback in the sense of a relaxed Bayesian consistency. We also prove that the empirical strategy profile induced by COL converges to the Berk-Nash equilibrium, a solution concept characterizing rationality under subjectivity. Experimental results from an intrusion response use case demonstrate COL's {faster convergence} over state-of-the-art reinforcement learning methods against nonstationary attacks.

Related papers

• FedMID: A Data-Free Method for Using Intermediate Outputs as a Defense Mechanism Against Poisoning Attacks in Federated Learning [17.796469530291954]
  Federated learning combines local updates from clients to produce a global model, which is susceptible to poisoning attacks. We present a new paradigm to defend against poisoning attacks in federated learning using functional mappings of local models based on intermediate outputs.
  arXiv  Detail & Related papers  (2024-04-18T05:10:05Z)
• Automated Security Response through Online Learning with Adaptive Conjectures [13.33996350474556]
  We study automated security response for an IT infrastructure. We formulate the interaction between an attacker and a defender as a partially observed, non-stationary game.
  arXiv  Detail & Related papers  (2024-02-19T20:06:15Z)
• Data Assimilation in Chaotic Systems Using Deep Reinforcement Learning [0.5999777817331317]
  Data assimilation plays a pivotal role in diverse applications, ranging from climate predictions and weather forecasts to trajectory planning for autonomous vehicles. Recent advancements have seen the emergence of deep learning approaches in this domain, primarily within a supervised learning framework. In this study, we introduce a novel DA strategy that utilizes reinforcement learning (RL) to apply state corrections using full or partial observations of the state variables.
  arXiv  Detail & Related papers  (2024-01-01T06:53:36Z)
• Scalable Learning of Intrusion Responses through Recursive Decomposition [0.0]
  We study automated intrusion response for an IT infrastructure and the interaction between an attacker and a defender as a partially observed game. To solve the game we follow an approach where attack and defense strategies co-evolve through reinforcement learning and self-play toward an equilibrium. We introduce an algorithm called Decompositional Fictitious Self-Play (DFSP), which learns equilibria through approximation.
  arXiv  Detail & Related papers  (2023-09-06T18:12:07Z)
• Towards Attack-tolerant Federated Learning via Critical Parameter Analysis [85.41873993551332]
  Federated learning systems are susceptible to poisoning attacks when malicious clients send false updates to the central server. This paper proposes a new defense strategy, FedCPA (Federated learning with Critical Analysis) Our attack-tolerant aggregation method is based on the observation that benign local models have similar sets of top-k and bottom-k critical parameters, whereas poisoned local models do not.
  arXiv  Detail & Related papers  (2023-08-18T05:37:55Z)
• Towards a Better Theoretical Understanding of Independent Subnetwork Training [56.24689348875711]
  We take a closer theoretical look at Independent Subnetwork Training (IST) IST is a recently proposed and highly effective technique for solving the aforementioned problems. We identify fundamental differences between IST and alternative approaches, such as distributed methods with compressed communication.
  arXiv  Detail & Related papers  (2023-06-28T18:14:22Z)
• A Linearly Convergent GAN Inversion-based Algorithm for Reverse Engineering of Deceptions [1.2891210250935146]
  We propose a novel framework for reverse engineering of deceptions that supposes that the clean data lies in the range of a GAN. For the first time in the literature, we provide deterministic linear convergence guarantees for this problem.
  arXiv  Detail & Related papers  (2023-06-07T20:08:27Z)
• Combating Exacerbated Heterogeneity for Robust Models in Federated Learning [91.88122934924435]
  Combination of adversarial training and federated learning can lead to the undesired robustness deterioration. We propose a novel framework called Slack Federated Adversarial Training (SFAT) We verify the rationality and effectiveness of SFAT on various benchmarked and real-world datasets.
  arXiv  Detail & Related papers  (2023-03-01T06:16:15Z)
• Adversarial Training with Complementary Labels: On the Benefit of Gradually Informative Attacks [119.38992029332883]
  Adversarial training with imperfect supervision is significant but receives limited attention. We propose a new learning strategy using gradually informative attacks. Experiments are conducted to demonstrate the effectiveness of our method on a range of benchmarked datasets.
  arXiv  Detail & Related papers  (2022-11-01T04:26:45Z)
• A Regularized Implicit Policy for Offline Reinforcement Learning [54.7427227775581]
  offline reinforcement learning enables learning from a fixed dataset, without further interactions with the environment. We propose a framework that supports learning a flexible yet well-regularized fully-implicit policy. Experiments and ablation study on the D4RL dataset validate our framework and the effectiveness of our algorithmic designs.
  arXiv  Detail & Related papers  (2022-02-19T20:22:04Z)
• Learning from Heterogeneous Data Based on Social Interactions over Graphs [58.34060409467834]
  This work proposes a decentralized architecture, where individual agents aim at solving a classification problem while observing streaming features of different dimensions. We show that the. strategy enables the agents to learn consistently under this highly-heterogeneous setting. We show that the. strategy enables the agents to learn consistently under this highly-heterogeneous setting.
  arXiv  Detail & Related papers  (2021-12-17T12:47:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.