ViNL: Visual Navigation and Locomotion Over Obstacles

• URL: http://arxiv.org/abs/2210.14791v3
• Date: Thu, 12 Oct 2023 19:10:57 GMT
• Title: ViNL: Visual Navigation and Locomotion Over Obstacles
• Authors: Simar Kareer, Naoki Yokoyama, Dhruv Batra, Sehoon Ha, Joanne Truong
• Abstract summary: We present Visual Navigation and Locomotion over obstacles (ViNL) It enables a quadrupedal robot to navigate unseen apartments while stepping over small obstacles that lie in its path. ViNL consists of: (1) a visual navigation policy that outputs linear and angular velocity commands that guides the robot to a goal coordinate in unfamiliar indoor environments; and (2) a visual locomotion policy that controls the robot's joints to avoid stepping on obstacles while following provided velocity commands.
• Score: 36.46953494419389
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present Visual Navigation and Locomotion over obstacles (ViNL), which enables a quadrupedal robot to navigate unseen apartments while stepping over small obstacles that lie in its path (e.g., shoes, toys, cables), similar to how humans and pets lift their feet over objects as they walk. ViNL consists of: (1) a visual navigation policy that outputs linear and angular velocity commands that guides the robot to a goal coordinate in unfamiliar indoor environments; and (2) a visual locomotion policy that controls the robot's joints to avoid stepping on obstacles while following provided velocity commands. Both the policies are entirely "model-free", i.e. sensors-to-actions neural networks trained end-to-end. The two are trained independently in two entirely different simulators and then seamlessly co-deployed by feeding the velocity commands from the navigator to the locomotor, entirely "zero-shot" (without any co-training). While prior works have developed learning methods for visual navigation or visual locomotion, to the best of our knowledge, this is the first fully learned approach that leverages vision to accomplish both (1) intelligent navigation in new environments, and (2) intelligent visual locomotion that aims to traverse cluttered environments without disrupting obstacles. On the task of navigation to distant goals in unknown environments, ViNL using just egocentric vision significantly outperforms prior work on robust locomotion using privileged terrain maps (+32.8% success and -4.42 collisions per meter). Additionally, we ablate our locomotion policy to show that each aspect of our approach helps reduce obstacle collisions. Videos and code at http://www.joannetruong.com/projects/vinl.html

Related papers

• Helpful DoggyBot: Open-World Object Fetching using Legged Robots and Vision-Language Models [63.89598561397856]
  We present a system for quadrupedal mobile manipulation in indoor environments. It uses a front-mounted gripper for object manipulation, a low-level controller trained in simulation using egocentric depth for agile skills. We evaluate our system in two unseen environments without any real-world data collection or training.
  arXiv  Detail & Related papers  (2024-09-30T20:58:38Z)
• DexDribbler: Learning Dexterous Soccer Manipulation via Dynamic Supervision [26.9579556496875]
  Joint manipulation of moving objects and locomotion with legs, such as playing soccer, receive scant attention in the learning community. We propose a feedback control block to compute the necessary body-level movement accurately and using the outputs as dynamic joint-level locomotion supervision. We observe that our learning scheme can not only make the policy network converge faster but also enable soccer robots to perform sophisticated maneuvers.
  arXiv  Detail & Related papers  (2024-03-21T11:16:28Z)
• Dexterous Legged Locomotion in Confined 3D Spaces with Reinforcement Learning [37.95557495560936]
  We introduce a hierarchical locomotion controller that combines a classical planner tasked with planning waypoints to reach a faraway global goal location, and an RL-based policy trained to follow these waypoints by generating low-level motion commands. In simulation, our hierarchical approach succeeds at navigating through demanding confined 3D environments, outperforming both pure end-to-end learning approaches and parameterized locomotion skills.
  arXiv  Detail & Related papers  (2024-03-06T16:49:08Z)
• Learning to Walk by Steering: Perceptive Quadrupedal Locomotion in Dynamic Environments [25.366480092589022]
  A quadrupedal robot must exhibit robust and agile walking behaviors in response to environmental clutter and moving obstacles. We present a hierarchical learning framework, named PRELUDE, which decomposes the problem of perceptive locomotion into high-level decision-making. We demonstrate the effectiveness of our approach in simulation and with hardware experiments.
  arXiv  Detail & Related papers  (2022-09-19T17:55:07Z)
• Coupling Vision and Proprioception for Navigation of Legged Robots [65.59559699815512]
  We exploit the complementary strengths of vision and proprioception to achieve point goal navigation in a legged robot. We show superior performance compared to wheeled robot (LoCoBot) baselines. We also show the real-world deployment of our system on a quadruped robot with onboard sensors and compute.
  arXiv  Detail & Related papers  (2021-12-03T18:59:59Z)
• Augmented reality navigation system for visual prosthesis [67.09251544230744]
  We propose an augmented reality navigation system for visual prosthesis that incorporates a software of reactive navigation and path planning. It consists on four steps: locating the subject on a map, planning the subject trajectory, showing it to the subject and re-planning without obstacles. Results show how our augmented navigation system help navigation performance by reducing the time and distance to reach the goals, even significantly reducing the number of obstacles collisions.
  arXiv  Detail & Related papers  (2021-09-30T09:41:40Z)
• Pushing it out of the Way: Interactive Visual Navigation [62.296686176988125]
  We study the problem of interactive navigation where agents learn to change the environment to navigate more efficiently to their goals. We introduce the Neural Interaction Engine (NIE) to explicitly predict the change in the environment caused by the agent's actions. By modeling the changes while planning, we find that agents exhibit significant improvements in their navigational capabilities.
  arXiv  Detail & Related papers  (2021-04-28T22:46:41Z)
• Learning Quadrupedal Locomotion over Challenging Terrain [68.51539602703662]
  Legged locomotion can dramatically expand the operational domains of robotics. Conventional controllers for legged locomotion are based on elaborate state machines that explicitly trigger the execution of motion primitives and reflexes. Here we present a radically robust controller for legged locomotion in challenging natural environments.
  arXiv  Detail & Related papers  (2020-10-21T19:11:20Z)
• Robot Perception enables Complex Navigation Behavior via Self-Supervised Learning [23.54696982881734]
  We propose an approach to unify successful robot perception systems for active target-driven navigation tasks via reinforcement learning (RL) Our method temporally incorporates compact motion and visual perception data, directly obtained using self-supervision from a single image sequence. We demonstrate our approach on two real-world driving dataset, KITTI and Oxford RobotCar, using the new interactive CityLearn framework.
  arXiv  Detail & Related papers  (2020-06-16T07:45:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.