Confidence-Aware Decision-Making and Control for Tool Selection

• URL: http://arxiv.org/abs/2403.03808v1
• Date: Wed, 6 Mar 2024 15:59:39 GMT
• Title: Confidence-Aware Decision-Making and Control for Tool Selection
• Authors: Ajith Anil Meera and Pablo Lanillos
• Abstract summary: Self-reflecting about our performance is essential for decision making. We introduce a mathematical framework that allows robots to use their control self-confidence to make better-informed decisions.
• Score: 1.550120821358415
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Self-reflecting about our performance (e.g., how confident we are) before doing a task is essential for decision making, such as selecting the most suitable tool or choosing the best route to drive. While this form of awareness -- thinking about our performance or metacognitive performance -- is well-known in humans, robots still lack this cognitive ability. This reflective monitoring can enhance their embodied decision power, robustness and safety. Here, we take a step in this direction by introducing a mathematical framework that allows robots to use their control self-confidence to make better-informed decisions. We derive a mathematical closed-form expression for control confidence for dynamic systems (i.e., the posterior inverse covariance of the control action). This control confidence seamlessly integrates within an objective function for decision making, that balances the: i) performance for task completion, ii) control effort, and iii) self-confidence. To evaluate our theoretical account, we framed the decision-making within the tool selection problem, where the agent has to select the best robot arm for a particular control task. The statistical analysis of the numerical simulations with randomized 2DOF arms shows that using control confidence during tool selection improves both real task performance, and the reliability of the tool for performance under unmodelled perturbations (e.g., external forces). Furthermore, our results indicate that control confidence is an early indicator of performance and thus, it can be used as a heuristic for making decisions when computation power is restricted or decision-making is intractable. Overall, we show the advantages of using confidence-aware decision-making and control scheme for dynamic systems.

Related papers

• Criticality and Safety Margins for Reinforcement Learning [53.10194953873209]
  We seek to define a criticality framework with both a quantifiable ground truth and a clear significance to users. We introduce true criticality as the expected drop in reward when an agent deviates from its policy for n consecutive random actions. We also introduce the concept of proxy criticality, a low-overhead metric that has a statistically monotonic relationship to true criticality.
  arXiv  Detail & Related papers  (2024-09-26T21:00:45Z)
• "A Good Bot Always Knows Its Limitations": Assessing Autonomous System Decision-making Competencies through Factorized Machine Self-confidence [5.167803438665586]
  Factorized Machine Self-confidence (FaMSeC) provides a holistic description of factors driving an algorithmic decision-making process. indicators are derived from hierarchical problem-solving statistics' embedded within broad classes of probabilistic decision-making algorithms. FaMSeC allows algorithmic goodness of fit' evaluations to be easily incorporated into the design of many kinds of autonomous agents.
  arXiv  Detail & Related papers  (2024-07-29T01:22:04Z)
• Automatic AI controller that can drive with confidence: steering vehicle with uncertainty knowledge [3.131134048419781]
  This research focuses on the development of a vehicle's lateral control system using a machine learning framework. We employ a Bayesian Neural Network (BNN), a probabilistic learning model, to address uncertainty quantification. By establishing a confidence threshold, we can trigger manual intervention, ensuring that control is relinquished from the algorithm when it operates outside of safe parameters.
  arXiv  Detail & Related papers  (2024-04-24T23:22:37Z)
• Rational Decision-Making Agent with Internalized Utility Judgment [91.80700126895927]
  Large language models (LLMs) have demonstrated remarkable advancements and have attracted significant efforts to develop LLMs into agents capable of executing intricate multi-step decision-making tasks beyond traditional NLP applications. This paper proposes RadAgent, which fosters the development of its rationality through an iterative framework involving Experience Exploration and Utility Learning. Experimental results on the ToolBench dataset demonstrate RadAgent's superiority over baselines, achieving over 10% improvement in Pass Rate on diverse tasks.
  arXiv  Detail & Related papers  (2023-08-24T03:11:45Z)
• Reinforcement Learning with a Terminator [80.34572413850186]
  We learn the parameters of the TerMDP and leverage the structure of the estimation problem to provide state-wise confidence bounds. We use these to construct a provably-efficient algorithm, which accounts for termination, and bound its regret.
  arXiv  Detail & Related papers  (2022-05-30T18:40:28Z)
• Inverse Online Learning: Understanding Non-Stationary and Reactionary Policies [79.60322329952453]
  We show how to develop interpretable representations of how agents make decisions. By understanding the decision-making processes underlying a set of observed trajectories, we cast the policy inference problem as the inverse to this online learning problem. We introduce a practical algorithm for retrospectively estimating such perceived effects, alongside the process through which agents update them. Through application to the analysis of UNOS organ donation acceptance decisions, we demonstrate that our approach can bring valuable insights into the factors that govern decision processes and how they change over time.
  arXiv  Detail & Related papers  (2022-03-14T17:40:42Z)
• Adaptive Autonomy in Human-on-the-Loop Vision-Based Robotics Systems [16.609594839630883]
  Computer vision approaches are widely used by autonomous robotic systems to guide their decision making. High accuracy is critical, particularly for Human-on-the-loop (HoTL) systems where humans play only a supervisory role. We propose a solution based upon adaptive autonomy levels, whereby the system detects loss of reliability of these models.
  arXiv  Detail & Related papers  (2021-03-28T05:43:10Z)
• Improving Robustness of Learning-based Autonomous Steering Using Adversarial Images [58.287120077778205]
  We introduce a framework for analyzing robustness of the learning algorithm w.r.t varying quality in the image input for autonomous driving. Using the results of sensitivity analysis, we propose an algorithm to improve the overall performance of the task of "learning to steer"
  arXiv  Detail & Related papers  (2021-02-26T02:08:07Z)
• Efficient Empowerment Estimation for Unsupervised Stabilization [75.32013242448151]
  empowerment principle enables unsupervised stabilization of dynamical systems at upright positions. We propose an alternative solution based on a trainable representation of a dynamical system as a Gaussian channel. We show that our method has a lower sample complexity, is more stable in training, possesses the essential properties of the empowerment function, and allows estimation of empowerment from images.
  arXiv  Detail & Related papers  (2020-07-14T21:10:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.