Conformalized Interactive Imitation Learning: Handling Expert Shift and Intermittent Feedback

• URL: http://arxiv.org/abs/2410.08852v2
• Date: Tue, 29 Apr 2025 12:17:52 GMT
• Title: Conformalized Interactive Imitation Learning: Handling Expert Shift and Intermittent Feedback
• Abstract summary: In interactive imitation learning (IL), uncertainty quantification offers a way for the learner (i.e. robot) to contend with distribution shifts encountered during deployment.<n>We develop ConformalDAgger, a new approach wherein the robot uses prediction intervals calibrated by IQT as a reliable measure of deployment-time uncertainty.<n>We find that in simulated and hardware deployments on a 7DOF robotic manipulator, ConformalDAgger detects high uncertainty when the expert shifts.
• Score: 26.87212135255959
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In interactive imitation learning (IL), uncertainty quantification offers a way for the learner (i.e. robot) to contend with distribution shifts encountered during deployment by actively seeking additional feedback from an expert (i.e. human) online. Prior works use mechanisms like ensemble disagreement or Monte Carlo dropout to quantify when black-box IL policies are uncertain; however, these approaches can lead to overconfident estimates when faced with deployment-time distribution shifts. Instead, we contend that we need uncertainty quantification algorithms that can leverage the expert human feedback received during deployment time to adapt the robot's uncertainty online. To tackle this, we draw upon online conformal prediction, a distribution-free method for constructing prediction intervals online given a stream of ground-truth labels. Human labels, however, are intermittent in the interactive IL setting. Thus, from the conformal prediction side, we introduce a novel uncertainty quantification algorithm called intermittent quantile tracking (IQT) that leverages a probabilistic model of intermittent labels, maintains asymptotic coverage guarantees, and empirically achieves desired coverage levels. From the interactive IL side, we develop ConformalDAgger, a new approach wherein the robot uses prediction intervals calibrated by IQT as a reliable measure of deployment-time uncertainty to actively query for more expert feedback. We compare ConformalDAgger to prior uncertainty-aware DAgger methods in scenarios where the distribution shift is (and isn't) present because of changes in the expert's policy. We find that in simulated and hardware deployments on a 7DOF robotic manipulator, ConformalDAgger detects high uncertainty when the expert shifts and increases the number of interventions compared to baselines, allowing the robot to more quickly learn the new behavior.

Related papers

• Online Conformal Prediction with Adversarial Semi-bandit Feedback via Regret Minimization [7.4627679994408735]
  Uncertainty quantification is crucial in safety-critical systems, where decisions must be made under uncertainty.<n>Online conformal prediction is a principled online uncertainty quantification method that dynamically constructs a prediction set at each time step.<n>We propose a novel learning method for online conformal prediction with partial feedback from an adaptive adversary.
  arXiv  Detail & Related papers  (2026-04-20T09:07:40Z)
• Uncertainty Awareness on Unsupervised Domain Adaptation for Time Series Data [49.36938105983916]
  Unsupervised domain adaptation methods seek to generalize effectively on unlabeled test data.<n>We propose incorporating multi-scale feature extraction and uncertainty estimation to improve the model's generalization and robustness across domains.
  arXiv  Detail & Related papers  (2025-08-26T03:13:08Z)
• Active Test-time Vision-Language Navigation [60.69722522420299]
  ATENA is a test-time active learning framework that enables a practical human-robot interaction via episodic feedback on uncertain navigation outcomes.<n>In particular, ATENA learns to increase certainty in successful episodes and decrease it in failed ones, improving uncertainty calibration.<n>In addition, we propose a self-active learning strategy that enables an agent to evaluate its navigation outcomes based on confident predictions.
  arXiv  Detail & Related papers  (2025-06-07T02:24:44Z)
• Error-quantified Conformal Inference for Time Series [40.438171912992864]
  Uncertainty quantification in time series prediction is challenging due to the temporal dependence and distribution shift on sequential data. We propose itError-quantified Conformal Inference (ECI) by smoothing the quantile loss function. ECI can achieve valid miscoverage control and output tighter prediction sets than other baselines.
  arXiv  Detail & Related papers  (2025-02-02T15:02:36Z)
• UAHOI: Uncertainty-aware Robust Interaction Learning for HOI Detection [18.25576487115016]
  This paper focuses on Human-Object Interaction (HOI) detection. It addresses the challenge of identifying and understanding the interactions between humans and objects within a given image or video frame. We propose a novel approach textscUAHOI, Uncertainty-aware Robust Human-Object Interaction Learning.
  arXiv  Detail & Related papers  (2024-08-14T10:06:39Z)
• The Over-Certainty Phenomenon in Modern Test-Time Adaptation Algorithms [8.210473195536077]
  We propose a solution that maintains accuracy and addresses calibration.<n>Our method achieves state-of-the-art performance in terms of Expected Error and Negative Log Likelihood.
  arXiv  Detail & Related papers  (2024-04-24T19:55:50Z)
• Uncertainty Quantification for Forward and Inverse Problems of PDEs via Latent Global Evolution [110.99891169486366]
  We propose a method that integrates efficient and precise uncertainty quantification into a deep learning-based surrogate model. Our method endows deep learning-based surrogate models with robust and efficient uncertainty quantification capabilities for both forward and inverse problems. Our method excels at propagating uncertainty over extended auto-regressive rollouts, making it suitable for scenarios involving long-term predictions.
  arXiv  Detail & Related papers  (2024-02-13T11:22:59Z)
• Lightweight, Uncertainty-Aware Conformalized Visual Odometry [2.429910016019183]
  Data-driven visual odometry (VO) is a critical subroutine for autonomous edge robotics. Emerging edge robotics devices like insect-scale drones and surgical robots lack a computationally efficient framework to estimate VO's predictive uncertainties. This paper presents a novel, lightweight, and statistically robust framework that leverages conformal inference (CI) to extract VO's uncertainty bands.
  arXiv  Detail & Related papers  (2023-03-03T20:37:55Z)
• Calibrating AI Models for Wireless Communications via Conformal Prediction [55.47458839587949]
  Conformal prediction is applied for the first time to the design of AI for communication systems. This paper investigates the application of conformal prediction as a general framework to obtain AI models that produce decisions with formal calibration guarantees.
  arXiv  Detail & Related papers  (2022-12-15T12:52:23Z)
• Interpretable Self-Aware Neural Networks for Robust Trajectory Prediction [50.79827516897913]
  We introduce an interpretable paradigm for trajectory prediction that distributes the uncertainty among semantic concepts. We validate our approach on real-world autonomous driving data, demonstrating superior performance over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2022-11-16T06:28:20Z)
• A general framework for multi-step ahead adaptive conformal heteroscedastic time series forecasting [0.0]
  This paper introduces a novel model-agnostic algorithm called adaptive ensemble batch multi-input multi-output conformalized quantile regression (AEnbMIMOCQR) It enables forecasters to generate multi-step ahead prediction intervals for a fixed pre-specified miscoverage rate in a distribution-free manner. Our method is grounded on conformal prediction principles, however, it does not require data splitting and provides close to exact coverage even when the data is not exchangeable.
  arXiv  Detail & Related papers  (2022-07-28T16:40:26Z)
• Probabilistic Symmetry for Multi-Agent Dynamics [18.94585103009698]
  We propose a novel deep dynamics model, Probabilistic Equivariant Continuous COnvolution (PECCO) for probabilistic prediction of multi-agent trajectories. PECCO shows significant improvements in accuracy and calibration compared to non-equivariant baselines.
  arXiv  Detail & Related papers  (2022-05-04T07:51:08Z)
• Training on Test Data with Bayesian Adaptation for Covariate Shift [96.3250517412545]
  Deep neural networks often make inaccurate predictions with unreliable uncertainty estimates. We derive a Bayesian model that provides for a well-defined relationship between unlabeled inputs under distributional shift and model parameters. We show that our method improves both accuracy and uncertainty estimation.
  arXiv  Detail & Related papers  (2021-09-27T01:09:08Z)
• A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification [1.90365714903665]
  This hands-on introduction is aimed at a reader interested in the practical implementation of distribution-free UQ. We will include many explanatory illustrations, examples, and code samples in Python, with PyTorch syntax.
  arXiv  Detail & Related papers  (2021-07-15T17:59:50Z)
• Learning Uncertainty For Safety-Oriented Semantic Segmentation In Autonomous Driving [77.39239190539871]
  We show how uncertainty estimation can be leveraged to enable safety critical image segmentation in autonomous driving. We introduce a new uncertainty measure based on disagreeing predictions as measured by a dissimilarity function. We show experimentally that our proposed approach is much less computationally intensive at inference time than competing methods.
  arXiv  Detail & Related papers  (2021-05-28T09:23:05Z)
• Quantifying Uncertainty in Deep Spatiotemporal Forecasting [67.77102283276409]
  We describe two types of forecasting problems: regular grid-based and graph-based. We analyze UQ methods from both the Bayesian and the frequentist point view, casting in a unified framework via statistical decision theory. Through extensive experiments on real-world road network traffic, epidemics, and air quality forecasting tasks, we reveal the statistical computational trade-offs for different UQ methods.
  arXiv  Detail & Related papers  (2021-05-25T14:35:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.