A Proprioceptive-Only Benchmark for Quadruped State Estimation: ATE, RPE, and Runtime Trade-offs Between Filters and Smoothers

• URL: http://arxiv.org/abs/2605.11674v1
• Date: Tue, 12 May 2026 07:31:57 GMT
• Title: A Proprioceptive-Only Benchmark for Quadruped State Estimation: ATE, RPE, and Runtime Trade-offs Between Filters and Smoothers
• Abstract summary: We compare three state-of-the-art proprioceptive state estimators for quadruped robots: MUSE, the Invariant Extended Kalman Filter (IEKF), and the Invariant Smoother (IS)<n>We report long-term accuracy (ATE), short-term accuracy (translational and rotational Relative Pose metrics RPE), and per-update time on a fixed hardware/software stack.<n>Overall, this study provides a concise snapshot of accuracy and cost, helping readers choose an estimator that fits their application constraints.
• Score: 3.926021278968717
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We compare three state-of-the-art proprioceptive state estimators for quadruped robots: MUSE [1], the Invariant Extended Kalman Filter (IEKF) [2], and the Invariant Smoother (IS) [3], on the CYN-1 sequence of the GrandTour Dataset [4]. Our goal is to give practitioners clear guidance on accuracy and computation time: we report long-term accuracy (Absolute Trajectory Error, ATE), short-term accuracy (translational and rotational Relative Pose Error, RPE), and per-update computation time on a fixed hardware/software stack. On this dataset, RPEs are broadly similar across methods, while IEKF and IS achieve a lower ATE than MUSE. Runtime results highlight the accuracy-latency trade-offs across the three approaches. In the discussion, we outline the evaluation choices used to ensure a fair comparison and analyze factors that influence short-horizon metrics. Overall, this study provides a concise snapshot of accuracy and cost, helping readers choose an estimator that fits their application constraints, with all evaluation code and documentation released open-source at https://github.com/iit-DLSLab/state_estimation_benchmark for full reproducibility.

Related papers

• Comparing Trajectories from Positions Alone: Curvature-Based Time Alignment and Drift Error Metric [9.234595526156854]
  In field robotics, acquiring independent large-scale reference trajectories more accurate than the evaluated estimates remains an open challenge.<n>The domain is widely reliant on Absolute Trajectory Error (ATE) and Relative Pose Error (RPE)<n>This paper introduces a trajectory-evaluation protocol for standardized and reliable accuracy assessment.
  arXiv  Detail & Related papers  (2026-09-14T02:30:33Z)
• Echo State Networks for Time Series Forecasting: Hyperparameter Sweep and Benchmarking [51.56484100374058]
  We evaluate whether a fully automatic, purely feedback-driven ESN can serve as a competitive alternative to widely used statistical forecasting methods.<n>Forecast accuracy is measured using MASE and sMAPE and benchmarked against simple benchmarks like drift and seasonal naive and statistical models.
  arXiv  Detail & Related papers  (2026-02-03T16:01:22Z)
• Rethinking Metrics and Benchmarks of Video Anomaly Detection [58.37571339811799]
  Video Anomaly Detection (VAD) aims to detect anomalies that deviate from expectation.<n>Existing VAD metrics are influenced by single annotation bias.<n>Existing benchmarks lack the capability to evaluate scene overfitting of fully/weakly-supervised algorithms.
  arXiv  Detail & Related papers  (2025-05-25T08:09:42Z)
• Temporal fine-tuning for early risk detection [3.8486074790756506]
  Early Risk Detection (ERD) on the Web aims to identify promptly users facing social and health issues.<n>Standard classification metrics may not suffice, resorting to specific metrics that explicitly consider precision and delay.<n>We propose a completely different strategy, temporal fine-tuning, which allows tuning transformer-based models by explicitly incorporating time.
  arXiv  Detail & Related papers  (2025-05-16T14:17:03Z)
• What is the Right Notion of Distance between Predict-then-Optimize Tasks? [35.842182348661076]
  OTD$3$ is a novel dataset distance that incorporates downstream decisions in addition to features and labels.<n>We show that our proposed distance accurately predicts model transferability across three different Predict-then- optimization tasks.
  arXiv  Detail & Related papers  (2024-09-11T04:13:17Z)
• A Closer Look at Debiased Temporal Sentence Grounding in Videos: Dataset, Metric, and Approach [53.727460222955266]
  Temporal Sentence Grounding in Videos (TSGV) aims to ground a natural language sentence in an untrimmed video. Recent studies have found that current benchmark datasets may have obvious moment annotation biases. We introduce a new evaluation metric "dR@n,IoU@m" that discounts the basic recall scores to alleviate the inflating evaluation caused by biased datasets.
  arXiv  Detail & Related papers  (2022-03-10T08:58:18Z)
• Leveraging Unlabeled Data to Predict Out-of-Distribution Performance [63.740181251997306]
  Real-world machine learning deployments are characterized by mismatches between the source (training) and target (test) distributions. In this work, we investigate methods for predicting the target domain accuracy using only labeled source data and unlabeled target data. We propose Average Thresholded Confidence (ATC), a practical method that learns a threshold on the model's confidence, predicting accuracy as the fraction of unlabeled examples.
  arXiv  Detail & Related papers  (2022-01-11T23:01:12Z)
• RIFLE: Imputation and Robust Inference from Low Order Marginals [10.082738539201804]
  We develop a statistical inference framework for regression and classification in the presence of missing data without imputation. Our framework, RIFLE, estimates low-order moments of the underlying data distribution with corresponding confidence intervals to learn a distributionally robust model. Our experiments demonstrate that RIFLE outperforms other benchmark algorithms when the percentage of missing values is high and/or when the number of data points is relatively small.
  arXiv  Detail & Related papers  (2021-09-01T23:17:30Z)
• VSAC: Efficient and Accurate Estimator for H and F [68.65610177368617]
  VSAC is a RANSAC-type robust estimator with a number of novelties. It is significantly faster than all its predecessors and runs on average in 1-2 ms, on a CPU. It is two orders of magnitude faster and yet as precise as MAGSAC++, the currently most accurate estimator of two-view geometry.
  arXiv  Detail & Related papers  (2021-06-18T17:04:57Z)
• TLIO: Tight Learned Inertial Odometry [43.17991168599939]
  We propose a tightly-coupled Extended Kalman Filter framework for IMU-only state estimation. We show that our network, trained with pedestrian data from a headset, can produce statistically consistent measurement and uncertainty.
  arXiv  Detail & Related papers  (2020-07-06T03:13:34Z)
• Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shift [81.74795324629712]
  We call prediction-time batch normalization, which significantly improves model accuracy and calibration under covariate shift. We show that prediction-time batch normalization provides complementary benefits to existing state-of-the-art approaches for improving robustness. The method has mixed results when used alongside pre-training, and does not seem to perform as well under more natural types of dataset shift.
  arXiv  Detail & Related papers  (2020-06-19T05:08:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.