Conformalized Teleoperation: Confidently Mapping Human Inputs to High-Dimensional Robot Actions

• URL: http://arxiv.org/abs/2406.07767v2
• Date: Wed, 10 Jul 2024 18:34:05 GMT
• Title: Conformalized Teleoperation: Confidently Mapping Human Inputs to High-Dimensional Robot Actions
• Authors: Michelle Zhao, Reid Simmons, Henny Admoni, Andrea Bajcsy,
• Abstract summary: We learn a mapping from low-dimensional human inputs to high-dimensional robot actions. Our key idea is to adapt the assistive map at training time to additionally estimate high-dimensional action quantiles. We propose an uncertainty-interval-based mechanism for detecting high-uncertainty user inputs and robot states.
• Score: 4.855534476454559
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Assistive robotic arms often have more degrees-of-freedom than a human teleoperator can control with a low-dimensional input, like a joystick. To overcome this challenge, existing approaches use data-driven methods to learn a mapping from low-dimensional human inputs to high-dimensional robot actions. However, determining if such a black-box mapping can confidently infer a user's intended high-dimensional action from low-dimensional inputs remains an open problem. Our key idea is to adapt the assistive map at training time to additionally estimate high-dimensional action quantiles, and then calibrate these quantiles via rigorous uncertainty quantification methods. Specifically, we leverage adaptive conformal prediction which adjusts the intervals over time, reducing the uncertainty bounds when the mapping is performant and increasing the bounds when the mapping consistently mis-predicts. Furthermore, we propose an uncertainty-interval-based mechanism for detecting high-uncertainty user inputs and robot states. We evaluate the efficacy of our proposed approach in a 2D assistive navigation task and two 7DOF Kinova Jaco tasks involving assistive cup grasping and goal reaching. Our findings demonstrate that conformalized assistive teleoperation manages to detect (but not differentiate between) high uncertainty induced by diverse preferences and induced by low-precision trajectories in the mapping's training dataset. On the whole, we see this work as a key step towards enabling robots to quantify their own uncertainty and proactively seek intervention when needed.

Related papers

• Social-Transmotion: Promptable Human Trajectory Prediction [65.80068316170613]
  Social-Transmotion is a generic Transformer-based model that exploits diverse and numerous visual cues to predict human behavior. Our approach is validated on multiple datasets, including JTA, JRDB, Pedestrians and Cyclists in Road Traffic, and ETH-UCY.
  arXiv  Detail & Related papers  (2023-12-26T18:56:49Z)
• AlphaBlock: Embodied Finetuning for Vision-Language Reasoning in Robot Manipulation [50.737355245505334]
  We propose a novel framework for learning high-level cognitive capabilities in robot manipulation tasks. The resulting dataset AlphaBlock consists of 35 comprehensive high-level tasks of multi-step text plans and paired observation.
  arXiv  Detail & Related papers  (2023-05-30T09:54:20Z)
• Distributional Instance Segmentation: Modeling Uncertainty and High Confidence Predictions with Latent-MaskRCNN [77.0623472106488]
  In this paper, we explore a class of distributional instance segmentation models using latent codes. For robotic picking applications, we propose a confidence mask method to achieve the high precision necessary. We show that our method can significantly reduce critical errors in robotic systems, including our newly released dataset of ambiguous scenes.
  arXiv  Detail & Related papers  (2023-05-03T05:57:29Z)
• Deep learning-based approaches for human motion decoding in smart walkers for rehabilitation [3.8791511769387634]
  Smart walkers should be able to decode human motion and needs, as early as possible. Current walkers decode motion intention using information of wearable or embedded sensors. A contactless approach is proposed, addressing human motion decoding as an early action recognition/detection problematic.
  arXiv  Detail & Related papers  (2023-01-13T14:29:44Z)
• Active Uncertainty Learning for Human-Robot Interaction: An Implicit Dual Control Approach [5.05828899601167]
  We present an algorithmic approach to enable uncertainty learning for human-in-the-loop motion planning based on the implicit dual control paradigm. Our approach relies on sampling-based approximation of dynamic programming model predictive control problem. The resulting policy is shown to preserve the dual control effect for generic human predictive models with both continuous and categorical uncertainty.
  arXiv  Detail & Related papers  (2022-02-15T20:40:06Z)
• SABER: Data-Driven Motion Planner for Autonomously Navigating Heterogeneous Robots [112.2491765424719]
  We present an end-to-end online motion planning framework that uses a data-driven approach to navigate a heterogeneous robot team towards a global goal. We use model predictive control (SMPC) to calculate control inputs that satisfy robot dynamics, and consider uncertainty during obstacle avoidance with chance constraints. recurrent neural networks are used to provide a quick estimate of future state uncertainty considered in the SMPC finite-time horizon solution. A Deep Q-learning agent is employed to serve as a high-level path planner, providing the SMPC with target positions that move the robots towards a desired global goal.
  arXiv  Detail & Related papers  (2021-08-03T02:56:21Z)
• Task-relevant Representation Learning for Networked Robotic Perception [74.0215744125845]
  This paper presents an algorithm to learn task-relevant representations of sensory data that are co-designed with a pre-trained robotic perception model's ultimate objective. Our algorithm aggressively compresses robotic sensory data by up to 11x more than competing methods.
  arXiv  Detail & Related papers  (2020-11-06T07:39:08Z)
• Autonomous Exploration Under Uncertainty via Deep Reinforcement Learning on Graphs [5.043563227694137]
  We consider an autonomous exploration problem in which a range-sensing mobile robot is tasked with accurately mapping the landmarks in an a priori unknown environment efficiently in real-time. We propose a novel approach that uses graph neural networks (GNNs) in conjunction with deep reinforcement learning (DRL), enabling decision-making over graphs containing exploration information to predict a robot's optimal sensing action in belief space.
  arXiv  Detail & Related papers  (2020-07-24T16:50:41Z)
• Understanding Multi-Modal Perception Using Behavioral Cloning for Peg-In-a-Hole Insertion Tasks [21.275342989110978]
  In this paper, we investigate the merits of multiple sensor modalities when combined to learn a controller for real world assembly operation tasks. We propose a multi-step-ahead loss function to improve the performance of the behavioral cloning method.
  arXiv  Detail & Related papers  (2020-07-22T19:46:51Z)
• Learning Compliance Adaptation in Contact-Rich Manipulation [81.40695846555955]
  We propose a novel approach for learning predictive models of force profiles required for contact-rich tasks. The approach combines an anomaly detection based on Bidirectional Gated Recurrent Units (Bi-GRU) and an adaptive force/impedance controller.
  arXiv  Detail & Related papers  (2020-05-01T05:23:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.