OptiState: State Estimation of Legged Robots using Gated Networks with Transformer-based Vision and Kalman Filtering

• URL: http://arxiv.org/abs/2401.16719v3
• Date: Sun, 28 Apr 2024 05:04:45 GMT
• Title: OptiState: State Estimation of Legged Robots using Gated Networks with Transformer-based Vision and Kalman Filtering
• Authors: Alexander Schperberg, Yusuke Tanaka, Saviz Mowlavi, Feng Xu, Bharathan Balaji, Dennis Hong,
• Abstract summary: State estimation for legged robots is challenging due to their highly dynamic motion and limitations imposed by sensor accuracy. We propose a hybrid solution that combines proprioception and exteroceptive information for estimating the state of the robot's trunk. This framework not only furnishes accurate robot state estimates, but can minimize the nonlinear errors that arise from sensor measurements and model simplifications through learning.
• Score: 42.817893456964
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: State estimation for legged robots is challenging due to their highly dynamic motion and limitations imposed by sensor accuracy. By integrating Kalman filtering, optimization, and learning-based modalities, we propose a hybrid solution that combines proprioception and exteroceptive information for estimating the state of the robot's trunk. Leveraging joint encoder and IMU measurements, our Kalman filter is enhanced through a single-rigid body model that incorporates ground reaction force control outputs from convex Model Predictive Control optimization. The estimation is further refined through Gated Recurrent Units, which also considers semantic insights and robot height from a Vision Transformer autoencoder applied on depth images. This framework not only furnishes accurate robot state estimates, including uncertainty evaluations, but can minimize the nonlinear errors that arise from sensor measurements and model simplifications through learning. The proposed methodology is evaluated in hardware using a quadruped robot on various terrains, yielding a 65% improvement on the Root Mean Squared Error compared to our VIO SLAM baseline. Code example: https://github.com/AlexS28/OptiState

Related papers

• Multi-Robot Relative Pose Estimation in SE(2) with Observability Analysis: A Comparison of Extended Kalman Filtering and Robust Pose Graph Optimization [1.0485739694839669]
  We focus on cooperative localization and observability analysis of relative pose estimation. For ROS/Gazebo simulations, we explore four sensing and communication structures. In hardware experiments, two Turtlebot3 equipped with UWB modules are used for real-world inter-robot relative pose estimation.
  arXiv  Detail & Related papers  (2024-01-27T06:09:56Z)
• Real-to-Sim: Predicting Residual Errors of Robotic Systems with Sparse Data using a Learning-based Unscented Kalman Filter [65.93205328894608]
  We learn the residual errors between a dynamic and/or simulator model and the real robot. We show that with the learned residual errors, we can further close the reality gap between dynamic models, simulations, and actual hardware.
  arXiv  Detail & Related papers  (2022-09-07T15:15:12Z)
• Visual-Inertial Odometry with Online Calibration of Velocity-Control Based Kinematic Motion Models [3.42658286826597]
  Visual-inertial odometry (VIO) is an important technology for autonomous robots with power and payload constraints. We propose a novel approach for VIO with stereo cameras which integrates and calibrates the velocity-control based kinematic motion model of wheeled mobile robots online.
  arXiv  Detail & Related papers  (2022-04-14T06:21:12Z)
• Visual-tactile sensing for Real-time liquid Volume Estimation in Grasping [58.50342759993186]
  We propose a visuo-tactile model for realtime estimation of the liquid inside a deformable container. We fuse two sensory modalities, i.e., the raw visual inputs from the RGB camera and the tactile cues from our specific tactile sensor. The robotic system is well controlled and adjusted based on the estimation model in real time.
  arXiv  Detail & Related papers  (2022-02-23T13:38:31Z)
• CNN-based Omnidirectional Object Detection for HermesBot Autonomous Delivery Robot with Preliminary Frame Classification [53.56290185900837]
  We propose an algorithm for optimizing a neural network for object detection using preliminary binary frame classification. An autonomous mobile robot with 6 rolling-shutter cameras on the perimeter providing a 360-degree field of view was used as the experimental setup.
  arXiv  Detail & Related papers  (2021-10-22T15:05:37Z)
• Robot Localization and Navigation through Predictive Processing using LiDAR [0.0]
  We show a proof-of-concept of the predictive processing-inspired approach to perception applied for localization and navigation using laser sensors. We learn the generative model of the laser through self-supervised learning and perform both online state-estimation and navigation. Results showed improved state-estimation performance when comparing to a state-of-the-art particle filter in the absence of odometry.
  arXiv  Detail & Related papers  (2021-09-09T09:58:00Z)
• Online Body Schema Adaptation through Cost-Sensitive Active Learning [63.84207660737483]
  The work was implemented in a simulation environment, using the 7DoF arm of the iCub robot simulator. A cost-sensitive active learning approach is used to select optimal joint configurations. The results show cost-sensitive active learning has similar accuracy to the standard active learning approach, while reducing in about half the executed movement.
  arXiv  Detail & Related papers  (2021-01-26T16:01:02Z)
• Robots State Estimation and Observability Analysis Based on Statistical Motion Models [10.941793802354953]
  This paper presents a generic motion model to capture mobile robots' dynamic behaviors (translation and rotation) The model is based on statistical models driven by white random processes and is formulated into a full state estimation algorithm based on the error-state extended Kalman filtering framework (ESEKF) A novel texttextbfitthin set concept is introduced to characterize the unobservable subset of the system states.
  arXiv  Detail & Related papers  (2020-10-12T18:35:33Z)
• Risk-Averse MPC via Visual-Inertial Input and Recurrent Networks for Online Collision Avoidance [95.86944752753564]
  We propose an online path planning architecture that extends the model predictive control (MPC) formulation to consider future location uncertainties. Our algorithm combines an object detection pipeline with a recurrent neural network (RNN) which infers the covariance of state estimates. The robustness of our methods is validated on complex quadruped robot dynamics and can be generally applied to most robotic platforms.
  arXiv  Detail & Related papers  (2020-07-28T07:34:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.