A Role of Environmental Complexity on Representation Learning in Deep Reinforcement Learning Agents

• URL: http://arxiv.org/abs/2407.03436v1
• Date: Wed, 3 Jul 2024 18:27:26 GMT
• Title: A Role of Environmental Complexity on Representation Learning in Deep Reinforcement Learning Agents
• Authors: Andrew Liu, Alla Borisyuk,
• Abstract summary: We developed a simulated navigation environment to train deep reinforcement learning agents. We modulated the frequency of exposure to a shortcut and navigation cue, leading to the development of artificial agents with differing abilities. We examined the encoded representations in artificial neural networks driving these agents, revealing intricate dynamics in representation learning.
• Score: 3.7314353481448337
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The environments where individuals live can present diverse navigation challenges, resulting in varying navigation abilities and strategies. Inspired by differing urban layouts and the Dual Solutions Paradigm test used for human navigators, we developed a simulated navigation environment to train deep reinforcement learning agents in a shortcut usage task. We modulated the frequency of exposure to a shortcut and navigation cue, leading to the development of artificial agents with differing abilities. We examined the encoded representations in artificial neural networks driving these agents, revealing intricate dynamics in representation learning, and correlated them with shortcut use preferences. Furthermore, we demonstrated methods to analyze representations across a population of nodes, which proved effective in finding patterns in what would otherwise be noisy single-node data. These techniques may also have broader applications in studying neural activity. From our observations in representation learning dynamics, we propose insights for human navigation learning, emphasizing the importance of navigation challenges in developing strong landmark knowledge over repeated exposures to landmarks alone.

Related papers

• A transformer-based deep reinforcement learning approach to spatial navigation in a partially observable Morris Water Maze [0.0]
  This work applies a transformer-based architecture using deep reinforcement learning to navigate a 2D version of the Morris Water Maze. We demonstrate that the proposed architecture enables the agent to efficiently learn spatial navigation strategies. This work suggests promising avenues for future research in artificial agents whose behavior resembles that of biological agents.
  arXiv  Detail & Related papers  (2024-10-01T13:22:56Z)
• Learning Navigational Visual Representations with Semantic Map Supervision [85.91625020847358]
  We propose a navigational-specific visual representation learning method by contrasting the agent's egocentric views and semantic maps. Ego$2$-Map learning transfers the compact and rich information from a map, such as objects, structure and transition, to the agent's egocentric representations for navigation.
  arXiv  Detail & Related papers  (2023-07-23T14:01:05Z)
• Investigating Navigation Strategies in the Morris Water Maze through Deep Reinforcement Learning [4.408196554639971]
  In this work, we simulate the Morris Water Maze in 2D to train deep reinforcement learning agents. We perform automatic classification of navigation strategies, analyze the distribution of strategies used by artificial agents, and compare them with experimental data to show similar learning dynamics as those seen in humans and rodents.
  arXiv  Detail & Related papers  (2023-06-01T18:16:16Z)
• Accelerating exploration and representation learning with offline pre-training [52.6912479800592]
  We show that exploration and representation learning can be improved by separately learning two different models from a single offline dataset. We show that learning a state representation using noise-contrastive estimation and a model of auxiliary reward can significantly improve the sample efficiency on the challenging NetHack benchmark.
  arXiv  Detail & Related papers  (2023-03-31T18:03:30Z)
• Multi-Object Navigation with dynamically learned neural implicit representations [10.182418917501064]
  We propose to structure neural networks with two neural implicit representations, which are learned dynamically during each episode. We evaluate the agent on Multi-Object Navigation and show the high impact of using neural implicit representations as a memory source.
  arXiv  Detail & Related papers  (2022-10-11T04:06:34Z)
• Backprop-Free Reinforcement Learning with Active Neural Generative Coding [84.11376568625353]
  We propose a computational framework for learning action-driven generative models without backpropagation of errors (backprop) in dynamic environments. We develop an intelligent agent that operates even with sparse rewards, drawing inspiration from the cognitive theory of planning as inference. The robust performance of our agent offers promising evidence that a backprop-free approach for neural inference and learning can drive goal-directed behavior.
  arXiv  Detail & Related papers  (2021-07-10T19:02:27Z)
• Deep Learning for Embodied Vision Navigation: A Survey [108.13766213265069]
  "Embodied visual navigation" problem requires an agent to navigate in a 3D environment mainly rely on its first-person observation. This paper attempts to establish an outline of the current works in the field of embodied visual navigation by providing a comprehensive literature survey.
  arXiv  Detail & Related papers  (2021-07-07T12:09:04Z)
• Causal Navigation by Continuous-time Neural Networks [108.84958284162857]
  We propose a theoretical and experimental framework for learning causal representations using continuous-time neural networks. We evaluate our method in the context of visual-control learning of drones over a series of complex tasks.
  arXiv  Detail & Related papers  (2021-06-15T17:45:32Z)
• Neural Topological SLAM for Visual Navigation [112.73876869904]
  We design topological representations for space that leverage semantics and afford approximate geometric reasoning. We describe supervised learning-based algorithms that can build, maintain and use such representations under noisy actuation.
  arXiv  Detail & Related papers  (2020-05-25T17:56:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.