VILENS: Visual, Inertial, Lidar, and Leg Odometry for All-Terrain Legged Robots

• URL: http://arxiv.org/abs/2107.07243v3
• Date: Mon, 07 Oct 2024 11:27:30 GMT
• Title: VILENS: Visual, Inertial, Lidar, and Leg Odometry for All-Terrain Legged Robots
• Authors: David Wisth, Marco Camurri, Maurice Fallon,
• Abstract summary: We present visual inertial lidar legged navigation system (VILENS) for legged robots. The key novelty is the tight fusion of four different sensor modalities to achieve reliable operation. We show an average improvement of 62% translational and 51% rotational errors compared to a state-of-the-art loosely coupled approach.
• Score: 5.789654849162465
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present visual inertial lidar legged navigation system (VILENS), an odometry system for legged robots based on factor graphs. The key novelty is the tight fusion of four different sensor modalities to achieve reliable operation when the individual sensors would otherwise produce degenerate estimation. To minimize leg odometry drift, we extend the robot's state with a linear velocity bias term, which is estimated online. This bias is observable because of the tight fusion of this preintegrated velocity factor with vision, lidar, and inertial measurement unit (IMU) factors. Extensive experimental validation on different ANYmal quadruped robots is presented, for a total duration of 2 h and 1.8 km traveled. The experiments involved dynamic locomotion over loose rocks, slopes, and mud, which caused challenges such as slippage and terrain deformation. Perceptual challenges included dark and dusty underground caverns, and open and feature-deprived areas. We show an average improvement of 62% translational and 51% rotational errors compared to a state-of-the-art loosely coupled approach. To demonstrate its robustness, VILENS was also integrated with a perceptive controller and a local path planner.

Related papers

• Estimating Scene Flow in Robot Surroundings with Distributed Miniaturized Time-of-Flight Sensors [41.45395153490076]
  We present an approach for scene flow estimation from low-density and noisy point clouds acquired from Time of Flight (ToF) sensors distributed on the robot body. The proposed method clusters points from consecutive frames and applies Iterative Closest Point (ICP) to estimate a dense motion flow. We employ a fitness-based classification to distinguish between stationary and moving points and an inlier removal strategy to refine geometric correspondences.
  arXiv  Detail & Related papers  (2025-04-03T09:57:51Z)
• Hybrid Internal Model: Learning Agile Legged Locomotion with Simulated Robot Response [25.52492911765911]
  We introduce Hybrid Internal Model to estimate external states according to the response of the robot. The response, which we refer to as the hybrid internal embedding, contains the robot's explicit velocity and implicit stability representation. A wealth of real-world experiments demonstrates its agility, even in high-difficulty tasks and cases never occurred during the training process.
  arXiv  Detail & Related papers  (2023-12-18T18:59:06Z)
• Tightly-coupled Visual-DVL-Inertial Odometry for Robot-based Ice-water Boundary Exploration [8.555466536537292]
  We present a multi-sensors fusion framework to increase localization accuracy. Visual images, Doppler Velocity Log (DVL), Inertial Measurement Unit (IMU) and Pressure sensor are integrated. The proposed method is validated with a data set collected in the field under frozen ice.
  arXiv  Detail & Related papers  (2023-03-29T20:16:39Z)
• Exploring Contextual Representation and Multi-Modality for End-to-End Autonomous Driving [58.879758550901364]
  Recent perception systems enhance spatial understanding with sensor fusion but often lack full environmental context. We introduce a framework that integrates three cameras to emulate the human field of view, coupled with top-down bird-eye-view semantic data to enhance contextual representation. Our method achieves displacement error by 0.67m in open-loop settings, surpassing current methods by 6.9% on the nuScenes dataset.
  arXiv  Detail & Related papers  (2022-10-13T05:56:20Z)
• Inertial Hallucinations -- When Wearable Inertial Devices Start Seeing Things [82.15959827765325]
  We propose a novel approach to multimodal sensor fusion for Ambient Assisted Living (AAL) We address two major shortcomings of standard multimodal approaches, limited area coverage and reduced reliability. Our new framework fuses the concept of modality hallucination with triplet learning to train a model with different modalities to handle missing sensors at inference time.
  arXiv  Detail & Related papers  (2022-07-14T10:04:18Z)
• VAE-Loco: Versatile Quadruped Locomotion by Learning a Disentangled Gait Representation [78.92147339883137]
  We show that it is pivotal in increasing controller robustness by learning a latent space capturing the key stance phases constituting a particular gait. We demonstrate that specific properties of the drive signal map directly to gait parameters such as cadence, footstep height and full stance duration. The use of a generative model facilitates the detection and mitigation of disturbances to provide a versatile and robust planning framework.
  arXiv  Detail & Related papers  (2022-05-02T19:49:53Z)
• Visual-tactile sensing for Real-time liquid Volume Estimation in Grasping [58.50342759993186]
  We propose a visuo-tactile model for realtime estimation of the liquid inside a deformable container. We fuse two sensory modalities, i.e., the raw visual inputs from the RGB camera and the tactile cues from our specific tactile sensor. The robotic system is well controlled and adjusted based on the estimation model in real time.
  arXiv  Detail & Related papers  (2022-02-23T13:38:31Z)
• Vision in adverse weather: Augmentation using CycleGANs with various object detectors for robust perception in autonomous racing [70.16043883381677]
  In autonomous racing, the weather can change abruptly, causing significant degradation in perception, resulting in ineffective manoeuvres. In order to improve detection in adverse weather, deep-learning-based models typically require extensive datasets captured in such conditions. We introduce an approach of using synthesised adverse condition datasets in autonomous racing (generated using CycleGAN) to improve the performance of four out of five state-of-the-art detectors.
  arXiv  Detail & Related papers  (2022-01-10T10:02:40Z)
• Unified Multi-Modal Landmark Tracking for Tightly Coupled Lidar-Visual-Inertial Odometry [5.131684964386192]
  We present an efficient multi-sensor odometry system for mobile platforms that jointly optimize visual, lidar, and inertial information. New method to extract 3D line and planar primitives from lidar point clouds is presented. System has been tested on a variety of platforms and scenarios, including underground exploration with a legged robot and outdoor scanning with a dynamically moving handheld device.
  arXiv  Detail & Related papers  (2020-11-13T09:54:03Z)
• PRGFlow: Benchmarking SWAP-Aware Unified Deep Visual Inertial Odometry [14.077054191270213]
  We present a deep learning approach for visual translation estimation and loosely fuse it with an Inertial sensor for full 6DoF odometry estimation. We evaluate our network on the MSCOCO dataset and evaluate the VI fusion on multiple real-flight trajectories.
  arXiv  Detail & Related papers  (2020-06-11T19:12:54Z)
• End-to-end Learning for Inter-Vehicle Distance and Relative Velocity Estimation in ADAS with a Monocular Camera [81.66569124029313]
  We propose a camera-based inter-vehicle distance and relative velocity estimation method based on end-to-end training of a deep neural network. The key novelty of our method is the integration of multiple visual clues provided by any two time-consecutive monocular frames. We also propose a vehicle-centric sampling mechanism to alleviate the effect of perspective distortion in the motion field.
  arXiv  Detail & Related papers  (2020-06-07T08:18:31Z)
• mmFall: Fall Detection using 4D MmWave Radar and a Hybrid Variational RNN AutoEncoder [0.4588028371034407]
  mmFall is an emerging millimeter-wave (mmWave) radar sensor to collect the human body's point cloud along with the body centroid. A fall is claimed to have occurred when the spike in anomaly level and the drop in centroid height occur simultaneously. To overcome the randomness in radar data, the proposed VRAE uses variational inference, a probabilistic approach rather than the traditional deterministic approach.
  arXiv  Detail & Related papers  (2020-03-05T00:37:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.