VOILA: Visual-Observation-Only Imitation Learning for Autonomous Navigation

• URL: http://arxiv.org/abs/2105.09371v1
• Date: Wed, 19 May 2021 19:25:23 GMT
• Title: VOILA: Visual-Observation-Only Imitation Learning for Autonomous Navigation
• Abstract summary: We introduce a new algorithm, Visual Observation only Imitation Learning for Autonomous navigation (VOILA) VOILA learns navigation policies from a single video demonstration collected from a physically different agent. We show that VOILA not only successfully imitates the expert, but that it also learns navigation policies that can generalize to novel environments.
• Score: 44.799092976955656
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While imitation learning for vision based autonomous mobile robot navigation has recently received a great deal of attention in the research community, existing approaches typically require state action demonstrations that were gathered using the deployment platform. However, what if one cannot easily outfit their platform to record these demonstration signals or worse yet the demonstrator does not have access to the platform at all? Is imitation learning for vision based autonomous navigation even possible in such scenarios? In this work, we hypothesize that the answer is yes and that recent ideas from the Imitation from Observation (IfO) literature can be brought to bear such that a robot can learn to navigate using only ego centric video collected by a demonstrator, even in the presence of viewpoint mismatch. To this end, we introduce a new algorithm, Visual Observation only Imitation Learning for Autonomous navigation (VOILA), that can successfully learn navigation policies from a single video demonstration collected from a physically different agent. We evaluate VOILA in the photorealistic AirSim simulator and show that VOILA not only successfully imitates the expert, but that it also learns navigation policies that can generalize to novel environments. Further, we demonstrate the effectiveness of VOILA in a real world setting by showing that it allows a wheeled Jackal robot to successfully imitate a human walking in an environment using a video recorded using a mobile phone camera.

Related papers

• SimWAM: A Simple World Action Model for End-to-End Autonomous Driving [66.93548079427477]
  We present SimWAM, a simple yet effective WAM that leverages future-video prediction as a training-time supervision signal.<n>An isolated attention mask keeps action prediction independent of future frames, allowing trajectory prediction without explicit future-frame generation at inference.<n>Our SimWAM achieves 91.5 PDMS on NAVSIM, surpasses state-of-the-art WAM-based planners with substantially lower latency, and transfers zero-shot to nuScenes.
  arXiv  Detail & Related papers  (2026-08-07T17:59:09Z)
• ImagiNav: Scalable Embodied Navigation via Generative Visual Prediction and Inverse Dynamics [13.382453086651019]
  Vision-Language Navigation has relied on end-to-end policies trained on expensive, embodiment-specific robot data.<n>We propose ImagiNav, a novel modular paradigm that decouples visual planning from robot actuation.<n>ImagiNav demonstrates strong zero-shot transfer to robot navigation without requiring robot demonstrations.
  arXiv  Detail & Related papers  (2026-03-14T08:34:02Z)
• Reflex-Based Open-Vocabulary Navigation without Prior Knowledge Using Omnidirectional Camera and Multiple Vision-Language Models [16.198686219364063]
  We consider the simplest method that does not require any map construction or learning, and execute open-vocabulary navigation of robots without any prior knowledge. We applied an omnidirectional camera and pre-trained vision-language models to the robot.
  arXiv  Detail & Related papers  (2024-08-21T07:18:58Z)
• Vision-based Manipulation from Single Human Video with Open-World Object Graphs [58.23098483464538]
  We present an object-centric approach to empower robots to learn vision-based manipulation skills from human videos. We introduce ORION, an algorithm that tackles the problem by extracting an object-centric manipulation plan from a single RGB-D video.
  arXiv  Detail & Related papers  (2024-05-30T17:56:54Z)
• LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action [76.71101507291473]
  We present a system, LM-Nav, for robotic navigation that enjoys the benefits of training on unannotated large datasets of trajectories. We show that such a system can be constructed entirely out of pre-trained models for navigation (ViNG), image-language association (CLIP), and language modeling (GPT-3), without requiring any fine-tuning or language-annotated robot data.
  arXiv  Detail & Related papers  (2022-07-10T10:41:50Z)
• NavDreams: Towards Camera-Only RL Navigation Among Humans [35.57943738219839]
  We investigate whether the world model concept, which has shown results for modeling and learning policies in Atari games, can also be applied to the camera-based navigation problem. We create simulated environments where a robot must navigate past static and moving humans without colliding in order to reach its goal. We find that state-of-the-art methods are able to achieve success in solving the navigation problem, and can generate dream-like predictions of future image-sequences.
  arXiv  Detail & Related papers  (2022-03-23T09:46:44Z)
• Rapid Exploration for Open-World Navigation with Latent Goal Models [78.45339342966196]
  We describe a robotic learning system for autonomous exploration and navigation in diverse, open-world environments. At the core of our method is a learned latent variable model of distances and actions, along with a non-parametric topological memory of images. We use an information bottleneck to regularize the learned policy, giving us (i) a compact visual representation of goals, (ii) improved generalization capabilities, and (iii) a mechanism for sampling feasible goals for exploration.
  arXiv  Detail & Related papers  (2021-04-12T23:14:41Z)
• ViNG: Learning Open-World Navigation with Visual Goals [82.84193221280216]
  We propose a learning-based navigation system for reaching visually indicated goals. We show that our system, which we call ViNG, outperforms previously-proposed methods for goal-conditioned reinforcement learning. We demonstrate ViNG on a number of real-world applications, such as last-mile delivery and warehouse inspection.
  arXiv  Detail & Related papers  (2020-12-17T18:22:32Z)
• LaND: Learning to Navigate from Disengagements [158.6392333480079]
  We present a reinforcement learning approach for learning to navigate from disengagements, or LaND. LaND learns a neural network model that predicts which actions lead to disengagements given the current sensory observation, and then at test time plans and executes actions that avoid disengagements. Our results demonstrate LaND can successfully learn to navigate in diverse, real world sidewalk environments, outperforming both imitation learning and reinforcement learning approaches.
  arXiv  Detail & Related papers  (2020-10-09T17:21:42Z)
• Visual Navigation Among Humans with Optimal Control as a Supervisor [72.5188978268463]
  We propose an approach that combines learning-based perception with model-based optimal control to navigate among humans. Our approach is enabled by our novel data-generation tool, HumANav. We demonstrate that the learned navigation policies can anticipate and react to humans without explicitly predicting future human motion.
  arXiv  Detail & Related papers  (2020-03-20T16:13:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.