Game-theoretic Objective Space Planning
- URL: http://arxiv.org/abs/2209.07758v1
- Date: Fri, 16 Sep 2022 07:35:20 GMT
- Title: Game-theoretic Objective Space Planning
- Authors: Hongrui Zheng, Zhijun Zhuang, Johannes Betz, Rahul Mangharam
- Abstract summary: Understanding intent of other agents is crucial to deploying autonomous systems in adversarial multi-agent environments.
Current approaches either oversimplify the discretization of the action space of agents or fail to recognize the long-term effect of actions and become myopic.
We propose a novel dimension reduction method that encapsulates diverse agent behaviors while conserving the continuity of agent actions.
- Score: 4.989480853499916
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Autonomous Racing awards agents that react to opponents' behaviors with agile
maneuvers towards progressing along the track while penalizing both
over-aggressive and over-conservative agents. Understanding the intent of other
agents is crucial to deploying autonomous systems in adversarial multi-agent
environments. Current approaches either oversimplify the discretization of the
action space of agents or fail to recognize the long-term effect of actions and
become myopic. Our work focuses on addressing these two challenges. First, we
propose a novel dimension reduction method that encapsulates diverse agent
behaviors while conserving the continuity of agent actions. Second, we
formulate the two-agent racing game as a regret minimization problem and
provide a solution for tractable counterfactual regret minimization with a
regret prediction model. Finally, we validate our findings experimentally on
scaled autonomous vehicles. We demonstrate that using the proposed
game-theoretic planner using agent characterization with the objective space
significantly improves the win rate against different opponents, and the
improvement is transferable to unseen opponents in an unseen environment.
Related papers
- Rethinking the Intermediate Features in Adversarial Attacks: Misleading Robotic Models via Adversarial Distillation [23.805401747928745]
This paper proposes a novel adversarial prompt attack tailored to language-conditioned robotic models.
We demonstrate that existing adversarial techniques exhibit limited effectiveness when directly transferred to the robotic domain.
We identify the beneficial impact of intermediate features on adversarial attacks and leverage the negative gradient of intermediate self-attention features to further enhance attack efficacy.
arXiv Detail & Related papers (2024-11-21T02:46:04Z) - Inverse Attention Agent for Multi-Agent System [6.196239958087161]
A major challenge for Multi-Agent Systems is enabling agents to adapt dynamically to diverse environments in which opponents and teammates may continually change.
We introduce Inverse Attention Agents that adopt concepts from the Theory of Mind, implemented algorithmically using an attention mechanism and trained in an end-to-end manner.
We demonstrate that the inverse attention network successfully infers the attention of other agents, and that this information improves agent performance.
arXiv Detail & Related papers (2024-10-29T06:59:11Z) - Improving the Generalization of Unseen Crowd Behaviors for Reinforcement Learning based Local Motion Planners [36.684452789236914]
Current Reinforcement Learning-based motion planners rely on a single policy to simulate pedestrian movements.
We introduce an efficient method that enhances agent diversity within a single policy by maximizing an information-theoretic objective.
In assessing an agent's robustness against unseen crowds, we propose diverse scenarios inspired by pedestrian crowd behaviors.
arXiv Detail & Related papers (2024-10-16T04:46:21Z) - Toward Optimal LLM Alignments Using Two-Player Games [86.39338084862324]
In this paper, we investigate alignment through the lens of two-agent games, involving iterative interactions between an adversarial and a defensive agent.
We theoretically demonstrate that this iterative reinforcement learning optimization converges to a Nash Equilibrium for the game induced by the agents.
Experimental results in safety scenarios demonstrate that learning in such a competitive environment not only fully trains agents but also leads to policies with enhanced generalization capabilities for both adversarial and defensive agents.
arXiv Detail & Related papers (2024-06-16T15:24:50Z) - Multi-granular Adversarial Attacks against Black-box Neural Ranking Models [111.58315434849047]
We create high-quality adversarial examples by incorporating multi-granular perturbations.
We transform the multi-granular attack into a sequential decision-making process.
Our attack method surpasses prevailing baselines in both attack effectiveness and imperceptibility.
arXiv Detail & Related papers (2024-04-02T02:08:29Z) - Formalizing the Problem of Side Effect Regularization [81.97441214404247]
We propose a formal criterion for side effect regularization via the assistance game framework.
In these games, the agent solves a partially observable Markov decision process.
We show that this POMDP is solved by trading off the proxy reward with the agent's ability to achieve a range of future tasks.
arXiv Detail & Related papers (2022-06-23T16:36:13Z) - Safe adaptation in multiagent competition [48.02377041620857]
In multiagent competitive scenarios, ego-agents may have to adapt to new opponents with previously unseen behaviors.
As the ego-agent updates its own behavior to exploit the opponent, its own behavior could become more exploitable.
We develop a safe adaptation approach in which the ego-agent is trained against a regularized opponent model.
arXiv Detail & Related papers (2022-03-14T23:53:59Z) - End-to-End Learning and Intervention in Games [60.41921763076017]
We provide a unified framework for learning and intervention in games.
We propose two approaches, respectively based on explicit and implicit differentiation.
The analytical results are validated using several real-world problems.
arXiv Detail & Related papers (2020-10-26T18:39:32Z) - Moody Learners -- Explaining Competitive Behaviour of Reinforcement
Learning Agents [65.2200847818153]
In a competitive scenario, the agent does not only have a dynamic environment but also is directly affected by the opponents' actions.
Observing the Q-values of the agent is usually a way of explaining its behavior, however, do not show the temporal-relation between the selected actions.
arXiv Detail & Related papers (2020-07-30T11:30:42Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.