Game-theoretic Objective Space Planning
- URL: http://arxiv.org/abs/2209.07758v1
- Date: Fri, 16 Sep 2022 07:35:20 GMT
- Title: Game-theoretic Objective Space Planning
- Authors: Hongrui Zheng, Zhijun Zhuang, Johannes Betz, Rahul Mangharam
- Abstract summary: Understanding intent of other agents is crucial to deploying autonomous systems in adversarial multi-agent environments.
Current approaches either oversimplify the discretization of the action space of agents or fail to recognize the long-term effect of actions and become myopic.
We propose a novel dimension reduction method that encapsulates diverse agent behaviors while conserving the continuity of agent actions.
- Score: 4.989480853499916
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Autonomous Racing awards agents that react to opponents' behaviors with agile
maneuvers towards progressing along the track while penalizing both
over-aggressive and over-conservative agents. Understanding the intent of other
agents is crucial to deploying autonomous systems in adversarial multi-agent
environments. Current approaches either oversimplify the discretization of the
action space of agents or fail to recognize the long-term effect of actions and
become myopic. Our work focuses on addressing these two challenges. First, we
propose a novel dimension reduction method that encapsulates diverse agent
behaviors while conserving the continuity of agent actions. Second, we
formulate the two-agent racing game as a regret minimization problem and
provide a solution for tractable counterfactual regret minimization with a
regret prediction model. Finally, we validate our findings experimentally on
scaled autonomous vehicles. We demonstrate that using the proposed
game-theoretic planner using agent characterization with the objective space
significantly improves the win rate against different opponents, and the
improvement is transferable to unseen opponents in an unseen environment.
Related papers
- Toward Optimal LLM Alignments Using Two-Player Games [86.39338084862324]
In this paper, we investigate alignment through the lens of two-agent games, involving iterative interactions between an adversarial and a defensive agent.
We theoretically demonstrate that this iterative reinforcement learning optimization converges to a Nash Equilibrium for the game induced by the agents.
Experimental results in safety scenarios demonstrate that learning in such a competitive environment not only fully trains agents but also leads to policies with enhanced generalization capabilities for both adversarial and defensive agents.
arXiv Detail & Related papers (2024-06-16T15:24:50Z) - Multi-granular Adversarial Attacks against Black-box Neural Ranking Models [111.58315434849047]
We create high-quality adversarial examples by incorporating multi-granular perturbations.
We transform the multi-granular attack into a sequential decision-making process.
Our attack method surpasses prevailing baselines in both attack effectiveness and imperceptibility.
arXiv Detail & Related papers (2024-04-02T02:08:29Z) - Mimicking To Dominate: Imitation Learning Strategies for Success in
Multiagent Competitive Games [13.060023718506917]
We develop a new multi-agent imitation learning model for predicting next moves of the opponents.
We also present a new multi-agent reinforcement learning algorithm that combines our imitation learning model and policy training into one single training process.
Experimental results show that our approach achieves superior performance compared to existing state-of-the-art multi-agent RL algorithms.
arXiv Detail & Related papers (2023-08-20T07:30:13Z) - Learning Models of Adversarial Agent Behavior under Partial
Observability [6.757727645540147]
We present Graph based Adversarial Modeling with Mutal Information (GrAMMI) for modeling the behavior of an adversarial opponent agent.
GrAMMI is a novel graph neural network (GNN) based approach that uses mutual information as an auxiliary objective.
arXiv Detail & Related papers (2023-06-19T21:18:52Z) - Formalizing the Problem of Side Effect Regularization [81.97441214404247]
We propose a formal criterion for side effect regularization via the assistance game framework.
In these games, the agent solves a partially observable Markov decision process.
We show that this POMDP is solved by trading off the proxy reward with the agent's ability to achieve a range of future tasks.
arXiv Detail & Related papers (2022-06-23T16:36:13Z) - Safe adaptation in multiagent competition [48.02377041620857]
In multiagent competitive scenarios, ego-agents may have to adapt to new opponents with previously unseen behaviors.
As the ego-agent updates its own behavior to exploit the opponent, its own behavior could become more exploitable.
We develop a safe adaptation approach in which the ego-agent is trained against a regularized opponent model.
arXiv Detail & Related papers (2022-03-14T23:53:59Z) - End-to-End Learning and Intervention in Games [60.41921763076017]
We provide a unified framework for learning and intervention in games.
We propose two approaches, respectively based on explicit and implicit differentiation.
The analytical results are validated using several real-world problems.
arXiv Detail & Related papers (2020-10-26T18:39:32Z) - Moody Learners -- Explaining Competitive Behaviour of Reinforcement
Learning Agents [65.2200847818153]
In a competitive scenario, the agent does not only have a dynamic environment but also is directly affected by the opponents' actions.
Observing the Q-values of the agent is usually a way of explaining its behavior, however, do not show the temporal-relation between the selected actions.
arXiv Detail & Related papers (2020-07-30T11:30:42Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.