Deep Reinforcement Learning for Navigation in AAA Video Games
- URL: http://arxiv.org/abs/2011.04764v2
- Date: Tue, 17 Nov 2020 19:09:57 GMT
- Title: Deep Reinforcement Learning for Navigation in AAA Video Games
- Authors: Eloi Alonso, Maxim Peter, David Goumard, Joshua Romoff
- Abstract summary: In video games, non-player characters (NPCs) are used to enhance the players' experience.
The most popular approach for NPC navigation in the video game industry is to use a navigation mesh (NavMesh)
We propose to use Deep Reinforcement Learning (Deep RL) to learn how to navigate 3D maps using any navigation ability.
- Score: 7.488317734152585
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In video games, non-player characters (NPCs) are used to enhance the players'
experience in a variety of ways, e.g., as enemies, allies, or innocent
bystanders. A crucial component of NPCs is navigation, which allows them to
move from one point to another on the map. The most popular approach for NPC
navigation in the video game industry is to use a navigation mesh (NavMesh),
which is a graph representation of the map, with nodes and edges indicating
traversable areas. Unfortunately, complex navigation abilities that extend the
character's capacity for movement, e.g., grappling hooks, jetpacks,
teleportation, or double-jumps, increases the complexity of the NavMesh, making
it intractable in many practical scenarios. Game designers are thus constrained
to only add abilities that can be handled by a NavMesh if they want to have NPC
navigation. As an alternative, we propose to use Deep Reinforcement Learning
(Deep RL) to learn how to navigate 3D maps using any navigation ability. We
test our approach on complex 3D environments in the Unity game engine that are
notably an order of magnitude larger than maps typically used in the Deep RL
literature. One of these maps is directly modeled after a Ubisoft AAA game. We
find that our approach performs surprisingly well, achieving at least $90\%$
success rate on all tested scenarios. A video of our results is available at
https://youtu.be/WFIf9Wwlq8M.
Related papers
- CityNav: Language-Goal Aerial Navigation Dataset with Geographic Information [25.51740922661166]
We introduce CityNav, a new dataset for language-goal aerial navigation using a 3D point cloud representation from real-world cities.
CityNav includes 32,637 natural language descriptions paired with human demonstration trajectories.
We benchmark the latest aerial navigation baselines and our proposed model on the CityNav dataset.
arXiv Detail & Related papers (2024-06-20T12:08:27Z) - GaussNav: Gaussian Splatting for Visual Navigation [92.13664084464514]
Instance ImageGoal Navigation (IIN) requires an agent to locate a specific object depicted in a goal image within an unexplored environment.
Our framework constructs a novel map representation based on 3D Gaussian Splatting (3DGS)
Our framework demonstrates a significant leap in performance, evidenced by an increase in Success weighted by Path Length (SPL) from 0.252 to 0.578 on the challenging Habitat-Matterport 3D (HM3D) dataset.
arXiv Detail & Related papers (2024-03-18T09:56:48Z) - Object Goal Navigation with Recursive Implicit Maps [92.6347010295396]
We propose an implicit spatial map for object goal navigation.
Our method significantly outperforms the state of the art on the challenging MP3D dataset.
We deploy our model on a real robot and achieve encouraging object goal navigation results in real scenes.
arXiv Detail & Related papers (2023-08-10T14:21:33Z) - SNAP: Self-Supervised Neural Maps for Visual Positioning and Semantic
Understanding [57.108301842535894]
We introduce SNAP, a deep network that learns rich neural 2D maps from ground-level and overhead images.
We train our model to align neural maps estimated from different inputs, supervised only with camera poses over tens of millions of StreetView images.
SNAP can resolve the location of challenging image queries beyond the reach of traditional methods.
arXiv Detail & Related papers (2023-06-08T17:54:47Z) - Go-Explore Complex 3D Game Environments for Automated Reachability
Testing [4.322647881761983]
We propose an approach specifically targeted at reachability bugs in simulated 3D environments based on the powerful exploration algorithm, Go-Explore.
Go-Explore saves unique checkpoints across the map and then identifies promising ones to explore from.
Our algorithm can fully cover a vast 1.5km x 1.5km game world within 10 hours on a single machine.
arXiv Detail & Related papers (2022-09-01T16:31:37Z) - CCPT: Automatic Gameplay Testing and Validation with
Curiosity-Conditioned Proximal Trajectories [65.35714948506032]
The Curiosity-Conditioned Proximal Trajectories (CCPT) method combines curiosity and imitation learning to train agents to explore.
We show how CCPT can explore complex environments, discover gameplay issues and design oversights in the process, and recognize and highlight them directly to game designers.
arXiv Detail & Related papers (2022-02-21T09:08:33Z) - Graph augmented Deep Reinforcement Learning in the GameRLand3D
environment [11.03710870581386]
We introduce a hybrid technique combining a low level policy trained with reinforcement learning and a graph based high level classical planner.
In an in-depth experimental study, we quantify the limitations of end-to-end Deep RL approaches in vast environments.
We also introduce "GameRLand3D", a new benchmark and soon to be released environment can generate complex procedural 3D maps for navigation tasks.
arXiv Detail & Related papers (2021-12-22T08:48:00Z) - MultiON: Benchmarking Semantic Map Memory using Multi-Object Navigation [23.877609358505268]
Recent work shows that map-like memory is useful for long-horizon navigation tasks.
We propose the multiON task, which requires navigation to an episode-specific sequence of objects in a realistic environment.
We examine how a variety of agent models perform across a spectrum of navigation task complexities.
arXiv Detail & Related papers (2020-12-07T18:42:38Z) - Active Visual Information Gathering for Vision-Language Navigation [115.40768457718325]
Vision-language navigation (VLN) is the task of entailing an agent to carry out navigational instructions inside photo-realistic environments.
One of the key challenges in VLN is how to conduct a robust navigation by mitigating the uncertainty caused by ambiguous instructions and insufficient observation of the environment.
This work draws inspiration from human navigation behavior and endows an agent with an active information gathering ability for a more intelligent VLN policy.
arXiv Detail & Related papers (2020-07-15T23:54:20Z) - Improving Target-driven Visual Navigation with Attention on 3D Spatial
Relationships [52.72020203771489]
We investigate target-driven visual navigation using deep reinforcement learning (DRL) in 3D indoor scenes.
Our proposed method combines visual features and 3D spatial representations to learn navigation policy.
Our experiments, performed in the AI2-THOR, show that our model outperforms the baselines in both SR and SPL metrics.
arXiv Detail & Related papers (2020-04-29T08:46:38Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.