Inverse Reinforcement Learning for Strategy Identification
- URL: http://arxiv.org/abs/2108.00293v1
- Date: Sat, 31 Jul 2021 17:22:52 GMT
- Title: Inverse Reinforcement Learning for Strategy Identification
- Authors: Mark Rucker, Stephen Adams, Roy Hayes, Peter A. Beling
- Abstract summary: In adversarial environments, one side could gain an advantage by identifying the opponent's strategy.
This paper proposes to use inverse reinforcement learning (IRL) to identify strategies in adversarial environments.
- Score: 2.6572330982240935
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: In adversarial environments, one side could gain an advantage by identifying
the opponent's strategy. For example, in combat games, if an opponents strategy
is identified as overly aggressive, one could lay a trap that exploits the
opponent's aggressive nature. However, an opponent's strategy is not always
apparent and may need to be estimated from observations of their actions. This
paper proposes to use inverse reinforcement learning (IRL) to identify
strategies in adversarial environments. Specifically, the contributions of this
work are 1) the demonstration of this concept on gaming combat data generated
from three pre-defined strategies and 2) the framework for using IRL to achieve
strategy identification. The numerical experiments demonstrate that the
recovered rewards can be identified using a variety of techniques. In this
paper, the recovered reward are visually displayed, clustered using
unsupervised learning, and classified using a supervised learner.
Related papers
- Learning in Markov Games with Adaptive Adversaries: Policy Regret, Fundamental Barriers, and Efficient Algorithms [24.928268203611047]
We study learning in a dynamically evolving environment modeled as a Markov game between a learner and a strategic opponent.
We focus on emphpolicy regret -- a counterfactual notion that aims to compete with the return that would have been attained if the learner had followed the best fixed sequence of policy.
arXiv Detail & Related papers (2024-11-01T16:17:27Z) - Learning Strategy Representation for Imitation Learning in Multi-Agent Games [15.209555810145549]
We introduce the Strategy Representation for Learning (STRIL) framework, which effectively learns strategy representations in multi-agent games.
STRIL is a plug-in method that can be integrated into existing IL algorithms.
We demonstrate the effectiveness of STRIL across competitive multi-agent scenarios, including Two-player Pong, Limit Texas Hold'em, and Connect Four.
arXiv Detail & Related papers (2024-09-28T14:30:17Z) - All by Myself: Learning Individualized Competitive Behaviour with a
Contrastive Reinforcement Learning optimization [57.615269148301515]
In a competitive game scenario, a set of agents have to learn decisions that maximize their goals and minimize their adversaries' goals at the same time.
We propose a novel model composed of three neural layers that learn a representation of a competitive game, learn how to map the strategy of specific opponents, and how to disrupt them.
Our experiments demonstrate that our model achieves better performance when playing against offline, online, and competitive-specific models, in particular when playing against the same opponent multiple times.
arXiv Detail & Related papers (2023-10-02T08:11:07Z) - Inverse-Inverse Reinforcement Learning. How to Hide Strategy from an
Adversarial Inverse Reinforcement Learner [19.044614610714856]
Inverse reinforcement learning deals with estimating an agent's utility function from its actions.
We consider how an agent can hide its strategy and mitigate an adversarial IRL attack.
arXiv Detail & Related papers (2022-05-22T11:54:44Z) - Projective Ranking-based GNN Evasion Attacks [52.85890533994233]
Graph neural networks (GNNs) offer promising learning methods for graph-related tasks.
GNNs are at risk of adversarial attacks.
arXiv Detail & Related papers (2022-02-25T21:52:09Z) - L2E: Learning to Exploit Your Opponent [66.66334543946672]
We propose a novel Learning to Exploit framework for implicit opponent modeling.
L2E acquires the ability to exploit opponents by a few interactions with different opponents during training.
We propose a novel opponent strategy generation algorithm that produces effective opponents for training automatically.
arXiv Detail & Related papers (2021-02-18T14:27:59Z) - Disturbing Reinforcement Learning Agents with Corrupted Rewards [62.997667081978825]
We analyze the effects of different attack strategies based on reward perturbations on reinforcement learning algorithms.
We show that smoothly crafting adversarial rewards are able to mislead the learner, and that using low exploration probability values, the policy learned is more robust to corrupt rewards.
arXiv Detail & Related papers (2021-02-12T15:53:48Z) - Incorporating Hidden Layer representation into Adversarial Attacks and
Defences [9.756797357009567]
We propose a defence strategy to improve adversarial robustness by incorporating hidden layer representation.
This strategy can be regarded as an activation function which can be applied to any kind of neural network.
arXiv Detail & Related papers (2020-11-28T01:41:57Z) - Learning to Play Sequential Games versus Unknown Opponents [93.8672371143881]
We consider a repeated sequential game between a learner, who plays first, and an opponent who responds to the chosen action.
We propose a novel algorithm for the learner when playing against an adversarial sequence of opponents.
Our results include algorithm's regret guarantees that depend on the regularity of the opponent's response.
arXiv Detail & Related papers (2020-07-10T09:33:05Z) - Efficient exploration of zero-sum stochastic games [83.28949556413717]
We investigate the increasingly important and common game-solving setting where we do not have an explicit description of the game but only oracle access to it through gameplay.
During a limited-duration learning phase, the algorithm can control the actions of both players in order to try to learn the game and how to play it well.
Our motivation is to quickly learn strategies that have low exploitability in situations where evaluating the payoffs of a queried strategy profile is costly.
arXiv Detail & Related papers (2020-02-24T20:30:38Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.