Learning to Move with Affordance Maps

• URL: http://arxiv.org/abs/2001.02364v2
• Date: Fri, 14 Feb 2020 19:01:26 GMT
• Title: Learning to Move with Affordance Maps
• Authors: William Qi, Ravi Teja Mullapudi, Saurabh Gupta, Deva Ramanan
• Abstract summary: The ability to autonomously explore and navigate a physical space is a fundamental requirement for virtually any mobile autonomous agent. Traditional SLAM-based approaches for exploration and navigation largely focus on leveraging scene geometry. We show that learned affordance maps can be used to augment traditional approaches for both exploration and navigation, providing significant improvements in performance.
• Score: 57.198806691838364
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The ability to autonomously explore and navigate a physical space is a fundamental requirement for virtually any mobile autonomous agent, from household robotic vacuums to autonomous vehicles. Traditional SLAM-based approaches for exploration and navigation largely focus on leveraging scene geometry, but fail to model dynamic objects (such as other agents) or semantic constraints (such as wet floors or doorways). Learning-based RL agents are an attractive alternative because they can incorporate both semantic and geometric information, but are notoriously sample inefficient, difficult to generalize to novel settings, and are difficult to interpret. In this paper, we combine the best of both worlds with a modular approach that learns a spatial representation of a scene that is trained to be effective when coupled with traditional geometric planners. Specifically, we design an agent that learns to predict a spatial affordance map that elucidates what parts of a scene are navigable through active self-supervised experience gathering. In contrast to most simulation environments that assume a static world, we evaluate our approach in the VizDoom simulator, using large-scale randomly-generated maps containing a variety of dynamic actors and hazards. We show that learned affordance maps can be used to augment traditional approaches for both exploration and navigation, providing significant improvements in performance.

Related papers

• E2Map: Experience-and-Emotion Map for Self-Reflective Robot Navigation with Language Models [16.50787220881633]
  Large language models (LLMs) have shown significant potential in guiding embodied agents to execute language instructions. Existing methods are primarily designed for static environments and do not leverage agent's own experiences to refine its initial plans. This study introduces the Experience-and-Emotion Map (E2Map), which not only integrates LLM knowledge but also the agent's real-world experiences.
  arXiv  Detail & Related papers  (2024-09-16T06:35:18Z)
• Multi-Object Navigation in real environments using hybrid policies [18.52681391843433]
  We introduce a hybrid navigation method, which decomposes the problem into two different skills. We show the advantages of this approach compared to end-to-end methods both in simulation and a real environment.
  arXiv  Detail & Related papers  (2024-01-24T20:41:25Z)
• Learning Navigational Visual Representations with Semantic Map Supervision [85.91625020847358]
  We propose a navigational-specific visual representation learning method by contrasting the agent's egocentric views and semantic maps. Ego$2$-Map learning transfers the compact and rich information from a map, such as objects, structure and transition, to the agent's egocentric representations for navigation.
  arXiv  Detail & Related papers  (2023-07-23T14:01:05Z)
• How To Not Train Your Dragon: Training-free Embodied Object Goal Navigation with Semantic Frontiers [94.46825166907831]
  We present a training-free solution to tackle the object goal navigation problem in Embodied AI. Our method builds a structured scene representation based on the classic visual simultaneous localization and mapping (V-SLAM) framework. Our method propagates semantics on the scene graphs based on language priors and scene statistics to introduce semantic knowledge to the geometric frontiers.
  arXiv  Detail & Related papers  (2023-05-26T13:38:33Z)
• Navigating to Objects in the Real World [76.1517654037993]
  We present a large-scale empirical study of semantic visual navigation methods comparing methods from classical, modular, and end-to-end learning approaches. We find that modular learning works well in the real world, attaining a 90% success rate. In contrast, end-to-end learning does not, dropping from 77% simulation to 23% real-world success rate due to a large image domain gap between simulation and reality.
  arXiv  Detail & Related papers  (2022-12-02T01:10:47Z)
• Bayesian Floor Field: Transferring people flow predictions across environments [8.691309696914882]
  We propose a novel approach to learn people dynamics able to combine knowledge about the environment geometry with observations from human trajectories. An occupancy-based deep prior is used to build an initial transition model without requiring any observations of pedestrian. We demonstrate the ability of our model to increase data efficiency and to generalize across real large-scale environments.
  arXiv  Detail & Related papers  (2022-08-23T10:09:40Z)
• TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors [74.67698916175614]
  We propose TrafficSim, a multi-agent behavior model for realistic traffic simulation. In particular, we leverage an implicit latent variable model to parameterize a joint actor policy. We show TrafficSim generates significantly more realistic and diverse traffic scenarios as compared to a diverse set of baselines.
  arXiv  Detail & Related papers  (2021-01-17T00:29:30Z)
• Robot Navigation in Constrained Pedestrian Environments using Reinforcement Learning [32.454250811667904]
  Navigating fluently around pedestrians is a necessary capability for mobile robots deployed in human environments. We present an approach based on reinforcement learning to learn policies capable of dynamic adaptation to the presence of moving pedestrians. We show transfer of the learned policy to unseen 3D reconstructions of two real environments.
  arXiv  Detail & Related papers  (2020-10-16T19:40:08Z)
• Occupancy Anticipation for Efficient Exploration and Navigation [97.17517060585875]
  We propose occupancy anticipation, where the agent uses its egocentric RGB-D observations to infer the occupancy state beyond the visible regions. By exploiting context in both the egocentric views and top-down maps our model successfully anticipates a broader map of the environment. Our approach is the winning entry in the 2020 Habitat PointNav Challenge.
  arXiv  Detail & Related papers  (2020-08-21T03:16:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.