Spot the Difference: A Novel Task for Embodied Agents in Changing Environments

• URL: http://arxiv.org/abs/2204.08502v1
• Date: Mon, 18 Apr 2022 18:30:56 GMT
• Title: Spot the Difference: A Novel Task for Embodied Agents in Changing Environments
• Authors: Federico Landi, Roberto Bigazzi, Marcella Cornia, Silvia Cascianelli, Lorenzo Baraldi and Rita Cucchiara
• Abstract summary: Embodied AI aims at creating intelligent agents that can move and operate inside an environment. We propose Spot the Difference: a novel task for Embodied AI where the agent has access to an outdated map of the environment. We propose an exploration policy that can take advantage of previous knowledge of the environment and identify changes in the scene faster and more effectively than existing agents.
• Score: 43.52107532692226
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Embodied AI is a recent research area that aims at creating intelligent agents that can move and operate inside an environment. Existing approaches in this field demand the agents to act in completely new and unexplored scenes. However, this setting is far from realistic use cases that instead require executing multiple tasks in the same environment. Even if the environment changes over time, the agent could still count on its global knowledge about the scene while trying to adapt its internal representation to the current state of the environment. To make a step towards this setting, we propose Spot the Difference: a novel task for Embodied AI where the agent has access to an outdated map of the environment and needs to recover the correct layout in a fixed time budget. To this end, we collect a new dataset of occupancy maps starting from existing datasets of 3D spaces and generating a number of possible layouts for a single environment. This dataset can be employed in the popular Habitat simulator and is fully compliant with existing methods that employ reconstructed occupancy maps during navigation. Furthermore, we propose an exploration policy that can take advantage of previous knowledge of the environment and identify changes in the scene faster and more effectively than existing agents. Experimental results show that the proposed architecture outperforms existing state-of-the-art models for exploration on this new setting.

Related papers

• Personalized Instance-based Navigation Toward User-Specific Objects in Realistic Environments [44.6372390798904]
  We propose a new task denominated Personalized Instance-based Navigation (PIN), in which an embodied agent is tasked with locating and reaching a specific personal object. In each episode, the target object is presented to the agent using two modalities: a set of visual reference images on a neutral background and manually annotated textual descriptions.
  arXiv  Detail & Related papers  (2024-10-23T18:01:09Z)
• AI planning in the imagination: High-level planning on learned abstract search spaces [68.75684174531962]
  We propose a new method, called PiZero, that gives an agent the ability to plan in an abstract search space that the agent learns during training. We evaluate our method on multiple domains, including the traveling salesman problem, Sokoban, 2048, the facility location problem, and Pacman.
  arXiv  Detail & Related papers  (2023-08-16T22:47:16Z)
• Open-ended search for environments and adapted agents using MAP-Elites [1.4502611532302039]
  We create a map of terrains and virtual creatures that locomote through them. By using novelty as a dimension in the grid, the map can continuously develop to encourage exploration of new environments. We find that while handcrafted bounded dimensions for the map lead to quicker exploration of a large set of environments, both the bounded and unbounded approach manage to solve a diverse set of terrains.
  arXiv  Detail & Related papers  (2023-05-02T02:03:51Z)
• CoDEPS: Online Continual Learning for Depth Estimation and Panoptic Segmentation [28.782231314289174]
  We introduce continual learning for deep learning-based monocular depth estimation and panoptic segmentation in new environments in an online manner. We propose a novel domain-mixing strategy to generate pseudo-labels to adapt panoptic segmentation. We explicitly address the limited storage capacity of robotic systems by leveraging sampling strategies for constructing a fixed-size replay buffer.
  arXiv  Detail & Related papers  (2023-03-17T17:31:55Z)
• EnvEdit: Environment Editing for Vision-and-Language Navigation [98.30038910061894]
  In Vision-and-Language Navigation (VLN), an agent needs to navigate through the environment based on natural language instructions. We propose EnvEdit, a data augmentation method that creates new environments by editing existing environments. We show that our proposed EnvEdit method gets significant improvements in all metrics on both pre-trained and non-pre-trained VLN agents.
  arXiv  Detail & Related papers  (2022-03-29T15:44:32Z)
• Rearrangement: A Challenge for Embodied AI [229.8891614821016]
  We describe a framework for research and evaluation in Embodied AI. Our proposal is based on a canonical task: Rearrangement. We present experimental testbeds of rearrangement scenarios in four different simulation environments.
  arXiv  Detail & Related papers  (2020-11-03T19:42:32Z)
• Self-Supervised Policy Adaptation during Deployment [98.25486842109936]
  Self-supervision allows the policy to continue training after deployment without using any rewards. Empirical evaluations are performed on diverse simulation environments from DeepMind Control suite and ViZDoom. Our method improves generalization in 31 out of 36 environments across various tasks and outperforms domain randomization on a majority of environments.
  arXiv  Detail & Related papers  (2020-07-08T17:56:27Z)
• An Exploration of Embodied Visual Exploration [97.21890864063872]
  Embodied computer vision considers perception for robots in novel, unstructured environments. We present a taxonomy for existing visual exploration algorithms and create a standard framework for benchmarking them. We then perform a thorough empirical study of the four state-of-the-art paradigms using the proposed framework.
  arXiv  Detail & Related papers  (2020-01-07T17:40:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.