Learning Perceptual Locomotion on Uneven Terrains using Sparse Visual Observations

• URL: http://arxiv.org/abs/2109.14026v1
• Date: Tue, 28 Sep 2021 20:25:10 GMT
• Title: Learning Perceptual Locomotion on Uneven Terrains using Sparse Visual Observations
• Authors: Fernando Acero, Kai Yuan, Zhibin Li
• Abstract summary: This work aims to exploit the use of sparse visual observations to achieve perceptual locomotion over a range of commonly seen bumps, ramps, and stairs in human-centred environments. We first formulate the selection of minimal visual input that can represent the uneven surfaces of interest, and propose a learning framework that integrates such exteroceptive and proprioceptive data. We validate the learned policy in tasks that require omnidirectional walking over flat ground and forward locomotion over terrains with obstacles, showing a high success rate.
• Score: 75.60524561611008
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Legged robots have achieved remarkable performance in blind walking using either model-based control or data-driven deep reinforcement learning. To proactively navigate and traverse various terrains, active use of visual perception becomes indispensable, and this work aims to exploit the use of sparse visual observations to achieve perceptual locomotion over a range of commonly seen bumps, ramps, and stairs in human-centred environments. We first formulate the selection of minimal visual input that can represent the uneven surfaces of interest, and propose a learning framework that integrates such exteroceptive and proprioceptive data. We specifically select state observations and design a training curriculum to learn feedback control policies more effectively over a range of different terrains. Using an extensive benchmark, we validate the learned policy in tasks that require omnidirectional walking over flat ground and forward locomotion over terrains with obstacles, showing a high success rate of traversal. Particularly, the robot performs autonomous perceptual locomotion with minimal visual perception using depth measurements, which are easily available from a Lidar or RGB-D sensor, and successfully demonstrates robust ascent and descent over high stairs of 20 cm step height, i.e., 50% of its leg length.

Related papers

• Human Insights Driven Latent Space for Different Driving Perspectives: A Unified Encoder for Efficient Multi-Task Inference [43.474068248379815]
  We propose a shared encoder trained on multiple computer vision tasks critical for urban navigation. We introduce a multi-scale feature network for pose estimation to improve depth learning. Our findings demonstrate that a shared backbone trained on diverse visual tasks is capable of providing overall perception capabilities.
  arXiv  Detail & Related papers  (2024-09-16T08:54:03Z)
• StairNet: Visual Recognition of Stairs for Human-Robot Locomotion [2.3811618212533663]
  StairNet is an initiative to support the development of new deep learning models for visual sensing and recognition of stairs. We present an overview of the development of our large-scale dataset with over 515,000 manually labeled images. We show that StairNet can be an effective platform to develop and study new visual perception systems for human-robot locomotion.
  arXiv  Detail & Related papers  (2023-10-31T17:30:57Z)
• Learning Navigational Visual Representations with Semantic Map Supervision [85.91625020847358]
  We propose a navigational-specific visual representation learning method by contrasting the agent's egocentric views and semantic maps. Ego$2$-Map learning transfers the compact and rich information from a map, such as objects, structure and transition, to the agent's egocentric representations for navigation.
  arXiv  Detail & Related papers  (2023-07-23T14:01:05Z)
• Fast Traversability Estimation for Wild Visual Navigation [17.015268056925745]
  We propose Wild Visual Navigation (WVN), an online self-supervised learning system for traversability estimation. The system is able to continuously adapt from a short human demonstration in the field. We demonstrate the advantages of our approach with experiments and ablation studies in challenging environments in forests, parks, and grasslands.
  arXiv  Detail & Related papers  (2023-05-15T10:19:30Z)
• Learning to See before Learning to Act: Visual Pre-training for Manipulation [48.731528716324355]
  We find that pre-training on vision tasks significantly improves generalization and sample efficiency for learning to manipulate objects. We explore directly transferring model parameters from vision networks to affordance prediction networks, and show that this can result in successful zero-shot adaptation. With just a small amount of robotic experience, we can further fine-tune the affordance model to achieve better results.
  arXiv  Detail & Related papers  (2021-07-01T17:58:37Z)
• Crop-Transform-Paste: Self-Supervised Learning for Visual Tracking [137.26381337333552]
  In this work, we develop the Crop-Transform-Paste operation, which is able to synthesize sufficient training data. Since the object state is known in all synthesized data, existing deep trackers can be trained in routine ways without human annotation.
  arXiv  Detail & Related papers  (2021-06-21T07:40:34Z)
• Rapid Exploration for Open-World Navigation with Latent Goal Models [78.45339342966196]
  We describe a robotic learning system for autonomous exploration and navigation in diverse, open-world environments. At the core of our method is a learned latent variable model of distances and actions, along with a non-parametric topological memory of images. We use an information bottleneck to regularize the learned policy, giving us (i) a compact visual representation of goals, (ii) improved generalization capabilities, and (iii) a mechanism for sampling feasible goals for exploration.
  arXiv  Detail & Related papers  (2021-04-12T23:14:41Z)
• What Can You Learn from Your Muscles? Learning Visual Representation from Human Interactions [50.435861435121915]
  We use human interaction and attention cues to investigate whether we can learn better representations compared to visual-only representations. Our experiments show that our "muscly-supervised" representation outperforms a visual-only state-of-the-art method MoCo.
  arXiv  Detail & Related papers  (2020-10-16T17:46:53Z)
• Robot Perception enables Complex Navigation Behavior via Self-Supervised Learning [23.54696982881734]
  We propose an approach to unify successful robot perception systems for active target-driven navigation tasks via reinforcement learning (RL) Our method temporally incorporates compact motion and visual perception data, directly obtained using self-supervision from a single image sequence. We demonstrate our approach on two real-world driving dataset, KITTI and Oxford RobotCar, using the new interactive CityLearn framework.
  arXiv  Detail & Related papers  (2020-06-16T07:45:47Z)
• Active Perception and Representation for Robotic Manipulation [0.8315801422499861]
  We present a framework that leverages the benefits of active perception to accomplish manipulation tasks. Our agent uses viewpoint changes to localize objects, to learn state representations in a self-supervised manner, and to perform goal-directed actions. Compared to vanilla deep Q-learning algorithms, our model is at least four times more sample-efficient.
  arXiv  Detail & Related papers  (2020-03-15T01:43:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.