Risk-aware Meta-level Decision Making for Exploration Under Uncertainty

• URL: http://arxiv.org/abs/2209.05580v3
• Date: Tue, 30 Apr 2024 15:38:46 GMT
• Title: Risk-aware Meta-level Decision Making for Exploration Under Uncertainty
• Authors: Joshua Ott, Sung-Kyun Kim, Amanda Bouman, Oriana Peltzer, Mamoru Sobue, Harrison Delecki, Mykel J. Kochenderfer, Joel Burdick, Ali-akbar Agha-mohammadi,
• Abstract summary: We propose a risk-aware meta-level decision making framework to balance the tradeoffs associated with local and global exploration. Our results show that by balancing local and global exploration we are able to significantly explore large-scale environments more efficiently.
• Score: 33.84356557132826
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Robotic exploration of unknown environments is fundamentally a problem of decision making under uncertainty where the robot must account for uncertainty in sensor measurements, localization, action execution, as well as many other factors. For large-scale exploration applications, autonomous systems must overcome the challenges of sequentially deciding which areas of the environment are valuable to explore while safely evaluating the risks associated with obstacles and hazardous terrain. In this work, we propose a risk-aware meta-level decision making framework to balance the tradeoffs associated with local and global exploration. Meta-level decision making builds upon classical hierarchical coverage planners by switching between local and global policies with the overall objective of selecting the policy that is most likely to maximize reward in a stochastic environment. We use information about the environment history, traversability risk, and kinodynamic constraints to reason about the probability of successful policy execution to switch between local and global policies. We have validated our solution in both simulation and on a variety of large-scale real world hardware tests. Our results show that by balancing local and global exploration we are able to significantly explore large-scale environments more efficiently.

Related papers

• Learning Where to Look: Self-supervised Viewpoint Selection for Active Localization using Geometrical Information [68.10033984296247]
  This paper explores the domain of active localization, emphasizing the importance of viewpoint selection to enhance localization accuracy. Our contributions involve using a data-driven approach with a simple architecture designed for real-time operation, a self-supervised data training method, and the capability to consistently integrate our map into a planning framework tailored for real-world robotics applications.
  arXiv  Detail & Related papers  (2024-07-22T12:32:09Z)
• Object Detectors in the Open Environment: Challenges, Solutions, and Outlook [95.3317059617271]
  The dynamic and intricate nature of the open environment poses novel and formidable challenges to object detectors. This paper aims to conduct a comprehensive review and analysis of object detectors in open environments. We propose a framework that includes four quadrants (i.e., out-of-domain, out-of-category, robust learning, and incremental learning) based on the dimensions of the data / target changes.
  arXiv  Detail & Related papers  (2024-03-24T19:32:39Z)
• HAZARD Challenge: Embodied Decision Making in Dynamically Changing Environments [93.94020724735199]
  HAZARD consists of three unexpected disaster scenarios, including fire, flood, and wind. This benchmark enables us to evaluate autonomous agents' decision-making capabilities across various pipelines.
  arXiv  Detail & Related papers  (2024-01-23T18:59:43Z)
• RiskBench: A Scenario-based Benchmark for Risk Identification [4.263035319815899]
  This work focuses on risk identification, the process of identifying and analyzing risks stemming from dynamic traffic participants and unexpected events. We introduce textbfRiskBench, a large-scale scenario-based benchmark for risk identification. We assess the ability of ten algorithms to (1) detect and locate risks, (2) anticipate risks, and (3) facilitate decision-making.
  arXiv  Detail & Related papers  (2023-12-04T06:21:22Z)
• Dealing with uncertainty: balancing exploration and exploitation in deep recurrent reinforcement learning [0.0]
  Incomplete knowledge of the environment leads an agent to make decisions under uncertainty. One of the major dilemmas in Reinforcement Learning (RL) where an autonomous agent has to balance two contrasting needs in making its decisions. We show that adaptive methods better approximate the trade-off between exploration and exploitation.
  arXiv  Detail & Related papers  (2023-10-12T13:45:33Z)
• Risk-Sensitive and Robust Model-Based Reinforcement Learning and Planning [2.627046865670577]
  We will address both planning and reinforcement learning approaches to sequential decision-making. In many real-world domains, it is impossible to construct a perfectly accurate model or simulator. We make a number of contributions towards this goal, with a focus on model-based algorithms.
  arXiv  Detail & Related papers  (2023-04-02T16:44:14Z)
• Sample-Based Bounds for Coherent Risk Measures: Applications to Policy Synthesis and Verification [32.9142708692264]
  This paper aims to address a few problems regarding risk-aware verification and policy synthesis. First, we develop a sample-based method to evaluate a subset of a random variable distribution. Second, we develop a robotic-based method to determine solutions to problems that outperform a large fraction of the decision space.
  arXiv  Detail & Related papers  (2022-04-21T01:06:10Z)
• Adaptive Risk Tendency: Nano Drone Navigation in Cluttered Environments with Distributional Reinforcement Learning [17.940958199767234]
  We present a distributional reinforcement learning framework to learn adaptive risk tendency policies. We show our algorithm can adjust its risk-sensitivity on the fly both in simulation and real-world experiments.
  arXiv  Detail & Related papers  (2022-03-28T13:39:58Z)
• MESA: Offline Meta-RL for Safe Adaptation and Fault Tolerance [73.3242641337305]
  Recent work learns risk measures which measure the probability of violating constraints, which can then be used to enable safety. We cast safe exploration as an offline meta-RL problem, where the objective is to leverage examples of safe and unsafe behavior across a range of environments. We then propose MEta-learning for Safe Adaptation (MESA), an approach for meta-learning Simulation a risk measure for safe RL.
  arXiv  Detail & Related papers  (2021-12-07T08:57:35Z)
• Cautious Adaptation For Reinforcement Learning in Safety-Critical Settings [129.80279257258098]
  Reinforcement learning (RL) in real-world safety-critical target settings like urban driving is hazardous. We propose a "safety-critical adaptation" task setting: an agent first trains in non-safety-critical "source" environments. We propose a solution approach, CARL, that builds on the intuition that prior experience in diverse environments equips an agent to estimate risk.
  arXiv  Detail & Related papers  (2020-08-15T01:40:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.