Open-ended search for environments and adapted agents using MAP-Elites

• URL: http://arxiv.org/abs/2305.01153v1
• Date: Tue, 2 May 2023 02:03:51 GMT
• Title: Open-ended search for environments and adapted agents using MAP-Elites
• Authors: Emma Stensby Norstein, Kai Olav Ellefsen, Kyrre Glette
• Abstract summary: We create a map of terrains and virtual creatures that locomote through them. By using novelty as a dimension in the grid, the map can continuously develop to encourage exploration of new environments. We find that while handcrafted bounded dimensions for the map lead to quicker exploration of a large set of environments, both the bounded and unbounded approach manage to solve a diverse set of terrains.
• Score: 1.4502611532302039
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Creatures in the real world constantly encounter new and diverse challenges they have never seen before. They will often need to adapt to some of these tasks and solve them in order to survive. This almost endless world of novel challenges is not as common in virtual environments, where artificially evolving agents often have a limited set of tasks to solve. An exception to this is the field of open-endedness where the goal is to create unbounded exploration of interesting artefacts. We want to move one step closer to creating simulated environments similar to the diverse real world, where agents can both find solvable tasks, and adapt to them. Through the use of MAP-Elites we create a structured repertoire, a map, of terrains and virtual creatures that locomote through them. By using novelty as a dimension in the grid, the map can continuously develop to encourage exploration of new environments. The agents must adapt to the environments found, but can also search for environments within each cell of the grid to find the one that best fits their set of skills. Our approach combines the structure of MAP-Elites, which can allow the virtual creatures to use adjacent cells as stepping stones to solve increasingly difficult environments, with open-ended innovation. This leads to a search that is unbounded, but still has a clear structure. We find that while handcrafted bounded dimensions for the map lead to quicker exploration of a large set of environments, both the bounded and unbounded approach manage to solve a diverse set of terrains.

Related papers

• ReALFRED: An Embodied Instruction Following Benchmark in Photo-Realistic Environments [13.988804095409133]
  We propose the ReALFRED benchmark that employs real-world scenes, objects, and room layouts to learn agents to complete household tasks. Specifically, we extend the ALFRED benchmark with updates for larger environmental spaces with smaller visual domain gaps. With ReALFRED, we analyze previously crafted methods for the ALFRED benchmark and observe that they consistently yield lower performance in all metrics.
  arXiv  Detail & Related papers  (2024-07-26T07:00:27Z)
• AI planning in the imagination: High-level planning on learned abstract search spaces [68.75684174531962]
  We propose a new method, called PiZero, that gives an agent the ability to plan in an abstract search space that the agent learns during training. We evaluate our method on multiple domains, including the traveling salesman problem, Sokoban, 2048, the facility location problem, and Pacman.
  arXiv  Detail & Related papers  (2023-08-16T22:47:16Z)
• Learning Hierarchical Interactive Multi-Object Search for Mobile Manipulation [10.21450780640562]
  We introduce a novel interactive multi-object search task in which a robot has to open doors to navigate rooms and search inside cabinets and drawers to find target objects. These new challenges require combining manipulation and navigation skills in unexplored environments. We present HIMOS, a hierarchical reinforcement learning approach that learns to compose exploration, navigation, and manipulation skills.
  arXiv  Detail & Related papers  (2023-07-12T12:25:33Z)
• CoDEPS: Online Continual Learning for Depth Estimation and Panoptic Segmentation [28.782231314289174]
  We introduce continual learning for deep learning-based monocular depth estimation and panoptic segmentation in new environments in an online manner. We propose a novel domain-mixing strategy to generate pseudo-labels to adapt panoptic segmentation. We explicitly address the limited storage capacity of robotic systems by leveraging sampling strategies for constructing a fixed-size replay buffer.
  arXiv  Detail & Related papers  (2023-03-17T17:31:55Z)
• WILD-SCAV: Benchmarking FPS Gaming AI on Unity3D-based Environments [5.020816812380825]
  Recent advances in deep reinforcement learning (RL) have demonstrated complex decision-making capabilities in simulation environments. However, they are hardly to more complicated problems, due to the lack of complexity and variations in the environments they are trained and tested on. We developed WILD-SCAV, a powerful and open-world environment based on a 3D open-world FPS game to bridge the gap. It provides realistic 3D environments of variable complexity, various tasks, and multiple modes of interaction, where agents can learn to perceive 3D environments, navigate and plan, compete and cooperate in a human-like manner
  arXiv  Detail & Related papers  (2022-10-14T13:39:41Z)
• Spot the Difference: A Novel Task for Embodied Agents in Changing Environments [43.52107532692226]
  Embodied AI aims at creating intelligent agents that can move and operate inside an environment. We propose Spot the Difference: a novel task for Embodied AI where the agent has access to an outdated map of the environment. We propose an exploration policy that can take advantage of previous knowledge of the environment and identify changes in the scene faster and more effectively than existing agents.
  arXiv  Detail & Related papers  (2022-04-18T18:30:56Z)
• EnvEdit: Environment Editing for Vision-and-Language Navigation [98.30038910061894]
  In Vision-and-Language Navigation (VLN), an agent needs to navigate through the environment based on natural language instructions. We propose EnvEdit, a data augmentation method that creates new environments by editing existing environments. We show that our proposed EnvEdit method gets significant improvements in all metrics on both pre-trained and non-pre-trained VLN agents.
  arXiv  Detail & Related papers  (2022-03-29T15:44:32Z)
• Fully Online Meta-Learning Without Task Boundaries [80.09124768759564]
  We study how meta-learning can be applied to tackle online problems of this nature. We propose a Fully Online Meta-Learning (FOML) algorithm, which does not require any ground truth knowledge about the task boundaries. Our experiments show that FOML was able to learn new tasks faster than the state-of-the-art online learning methods.
  arXiv  Detail & Related papers  (2022-02-01T07:51:24Z)
• Learning Affordance Landscapes for Interaction Exploration in 3D Environments [101.90004767771897]
  Embodied agents must be able to master how their environment works. We introduce a reinforcement learning approach for exploration for interaction. We demonstrate our idea with AI2-iTHOR.
  arXiv  Detail & Related papers  (2020-08-21T00:29:36Z)
• Learning to Move with Affordance Maps [57.198806691838364]
  The ability to autonomously explore and navigate a physical space is a fundamental requirement for virtually any mobile autonomous agent. Traditional SLAM-based approaches for exploration and navigation largely focus on leveraging scene geometry. We show that learned affordance maps can be used to augment traditional approaches for both exploration and navigation, providing significant improvements in performance.
  arXiv  Detail & Related papers  (2020-01-08T04:05:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.