Deep Reinforcement Learning for Navigation in AAA Video Games

• URL: http://arxiv.org/abs/2011.04764v2
• Date: Tue, 17 Nov 2020 19:09:57 GMT
• Title: Deep Reinforcement Learning for Navigation in AAA Video Games
• Authors: Eloi Alonso, Maxim Peter, David Goumard, Joshua Romoff
• Abstract summary: In video games, non-player characters (NPCs) are used to enhance the players' experience. The most popular approach for NPC navigation in the video game industry is to use a navigation mesh (NavMesh) We propose to use Deep Reinforcement Learning (Deep RL) to learn how to navigate 3D maps using any navigation ability.
• Score: 7.488317734152585
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In video games, non-player characters (NPCs) are used to enhance the players' experience in a variety of ways, e.g., as enemies, allies, or innocent bystanders. A crucial component of NPCs is navigation, which allows them to move from one point to another on the map. The most popular approach for NPC navigation in the video game industry is to use a navigation mesh (NavMesh), which is a graph representation of the map, with nodes and edges indicating traversable areas. Unfortunately, complex navigation abilities that extend the character's capacity for movement, e.g., grappling hooks, jetpacks, teleportation, or double-jumps, increases the complexity of the NavMesh, making it intractable in many practical scenarios. Game designers are thus constrained to only add abilities that can be handled by a NavMesh if they want to have NPC navigation. As an alternative, we propose to use Deep Reinforcement Learning (Deep RL) to learn how to navigate 3D maps using any navigation ability. We test our approach on complex 3D environments in the Unity game engine that are notably an order of magnitude larger than maps typically used in the Deep RL literature. One of these maps is directly modeled after a Ubisoft AAA game. We find that our approach performs surprisingly well, achieving at least $90\%$ success rate on all tested scenarios. A video of our results is available at https://youtu.be/WFIf9Wwlq8M.

Related papers

• Training Interactive Agent in Large FPS Game Map with Rule-enhanced Reinforcement Learning [10.637376058491224]
  We focus on the practical deployment of game AI in the online multiplayer competitive 3D FPS game called Arena Breakout, developed by Tencent Games. We propose a novel gaming AI system named Private Military Company Agent (PMCA), which is interactable within a large game map. To address the challenges of navigation and combat in modern 3D FPS games, we introduce a method that combines navigation mesh (Navmesh) and shooting-rule with deep reinforcement learning (NSRL)
  arXiv  Detail & Related papers  (2024-10-07T11:27:45Z)
• Hyp2Nav: Hyperbolic Planning and Curiosity for Crowd Navigation [58.574464340559466]
  We advocate for hyperbolic learning to enable crowd navigation and we introduce Hyp2Nav. Hyp2Nav leverages the intrinsic properties of hyperbolic geometry to better encode the hierarchical nature of decision-making processes in navigation tasks. We propose a hyperbolic policy model and a hyperbolic curiosity module that results in effective social navigation, best success rates, and returns across multiple simulation settings.
  arXiv  Detail & Related papers  (2024-07-18T14:40:33Z)
• CityNav: Language-Goal Aerial Navigation Dataset with Geographic Information [25.51740922661166]
  Vision-and-language navigation (VLN) aims to guide autonomous agents through real-world environments by integrating visual and linguistic cues. We introduce CityNav, a novel dataset explicitly designed for language-guided aerial navigation in 3D environments of real cities. CityNav comprises 32k natural language descriptions paired with human demonstration trajectories, collected via a newly developed web-based 3D simulator.
  arXiv  Detail & Related papers  (2024-06-20T12:08:27Z)
• GaussNav: Gaussian Splatting for Visual Navigation [92.13664084464514]
  Instance ImageGoal Navigation (IIN) requires an agent to locate a specific object depicted in a goal image within an unexplored environment. Our framework constructs a novel map representation based on 3D Gaussian Splatting (3DGS) Our framework demonstrates a significant leap in performance, evidenced by an increase in Success weighted by Path Length (SPL) from 0.252 to 0.578 on the challenging Habitat-Matterport 3D (HM3D) dataset.
  arXiv  Detail & Related papers  (2024-03-18T09:56:48Z)
• Object Goal Navigation with Recursive Implicit Maps [92.6347010295396]
  We propose an implicit spatial map for object goal navigation. Our method significantly outperforms the state of the art on the challenging MP3D dataset. We deploy our model on a real robot and achieve encouraging object goal navigation results in real scenes.
  arXiv  Detail & Related papers  (2023-08-10T14:21:33Z)
• Go-Explore Complex 3D Game Environments for Automated Reachability Testing [4.322647881761983]
  We propose an approach specifically targeted at reachability bugs in simulated 3D environments based on the powerful exploration algorithm, Go-Explore. Go-Explore saves unique checkpoints across the map and then identifies promising ones to explore from. Our algorithm can fully cover a vast 1.5km x 1.5km game world within 10 hours on a single machine.
  arXiv  Detail & Related papers  (2022-09-01T16:31:37Z)
• CCPT: Automatic Gameplay Testing and Validation with Curiosity-Conditioned Proximal Trajectories [65.35714948506032]
  The Curiosity-Conditioned Proximal Trajectories (CCPT) method combines curiosity and imitation learning to train agents to explore. We show how CCPT can explore complex environments, discover gameplay issues and design oversights in the process, and recognize and highlight them directly to game designers.
  arXiv  Detail & Related papers  (2022-02-21T09:08:33Z)
• Graph augmented Deep Reinforcement Learning in the GameRLand3D environment [11.03710870581386]
  We introduce a hybrid technique combining a low level policy trained with reinforcement learning and a graph based high level classical planner. In an in-depth experimental study, we quantify the limitations of end-to-end Deep RL approaches in vast environments. We also introduce "GameRLand3D", a new benchmark and soon to be released environment can generate complex procedural 3D maps for navigation tasks.
  arXiv  Detail & Related papers  (2021-12-22T08:48:00Z)
• Active Visual Information Gathering for Vision-Language Navigation [115.40768457718325]
  Vision-language navigation (VLN) is the task of entailing an agent to carry out navigational instructions inside photo-realistic environments. One of the key challenges in VLN is how to conduct a robust navigation by mitigating the uncertainty caused by ambiguous instructions and insufficient observation of the environment. This work draws inspiration from human navigation behavior and endows an agent with an active information gathering ability for a more intelligent VLN policy.
  arXiv  Detail & Related papers  (2020-07-15T23:54:20Z)
• Improving Target-driven Visual Navigation with Attention on 3D Spatial Relationships [52.72020203771489]
  We investigate target-driven visual navigation using deep reinforcement learning (DRL) in 3D indoor scenes. Our proposed method combines visual features and 3D spatial representations to learn navigation policy. Our experiments, performed in the AI2-THOR, show that our model outperforms the baselines in both SR and SPL metrics.
  arXiv  Detail & Related papers  (2020-04-29T08:46:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.