When to Ask for Help: Proactive Interventions in Autonomous Reinforcement Learning

• URL: http://arxiv.org/abs/2210.10765v1
• Date: Wed, 19 Oct 2022 17:57:24 GMT
• Title: When to Ask for Help: Proactive Interventions in Autonomous Reinforcement Learning
• Authors: Annie Xie, Fahim Tajwar, Archit Sharma, Chelsea Finn
• Abstract summary: A long-term goal of reinforcement learning is to design agents that can autonomously interact and learn in the world. A critical challenge is the presence of irreversible states which require external assistance to recover from, such as when a robot arm has pushed an object off of a table. We propose an algorithm that efficiently learns to detect and avoid states that are irreversible, and proactively asks for help in case the agent does enter them.
• Score: 57.53138994155612
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A long-term goal of reinforcement learning is to design agents that can autonomously interact and learn in the world. A critical challenge to such autonomy is the presence of irreversible states which require external assistance to recover from, such as when a robot arm has pushed an object off of a table. While standard agents require constant monitoring to decide when to intervene, we aim to design proactive agents that can request human intervention only when needed. To this end, we propose an algorithm that efficiently learns to detect and avoid states that are irreversible, and proactively asks for help in case the agent does enter them. On a suite of continuous control environments with unknown irreversible states, we find that our algorithm exhibits better sample- and intervention-efficiency compared to existing methods. Our code is publicly available at https://sites.google.com/view/proactive-interventions

Related papers

• Get the Ball Rolling: Alerting Autonomous Robots When to Help to Close the Healthcare Loop [25.551355056830413]
  We introduce the Autonomous Helping Challenge, along with a crowd-sourcing large-scale dataset. The goal is to create healthcare robots that possess the ability to determine when assistance is necessary. We propose Helpy, a potential approach to close the healthcare loop in the learning-free setting.
  arXiv  Detail & Related papers  (2023-11-05T08:57:59Z)
• Conveying Autonomous Robot Capabilities through Contrasting Behaviour Summaries [8.413049356622201]
  We present an adaptive search method for efficiently generating contrasting behaviour summaries. Our results indicate that adaptive search can efficiently identify informative contrasting scenarios that enable humans to accurately select the better performing agent.
  arXiv  Detail & Related papers  (2023-04-01T18:20:59Z)
• Decision Making for Human-in-the-loop Robotic Agents via Uncertainty-Aware Reinforcement Learning [13.184897303302971]
  In a Human-in-the-Loop paradigm, a robotic agent is able to act mostly autonomously in solving a task, but can request help from an external expert when needed. We present a Reinforcement Learning based approach to this problem, where a semi-autonomous agent asks for external assistance when it has low confidence in the eventual success of the task. We show that our method makes effective use of a limited budget of expert calls at run-time, despite having no access to the expert at training time.
  arXiv  Detail & Related papers  (2023-03-12T17:22:54Z)
• Persistent Reinforcement Learning via Subgoal Curricula [114.83989499740193]
  Value-accelerated Persistent Reinforcement Learning (VaPRL) generates a curriculum of initial states. VaPRL reduces the interventions required by three orders of magnitude compared to episodic reinforcement learning.
  arXiv  Detail & Related papers  (2021-07-27T16:39:45Z)
• Human-in-the-Loop Imitation Learning using Remote Teleoperation [72.2847988686463]
  We build a data collection system tailored to 6-DoF manipulation settings. We develop an algorithm to train the policy iteratively on new data collected by the system. We demonstrate that agents trained on data collected by our intervention-based system and algorithm outperform agents trained on an equivalent number of samples collected by non-interventional demonstrators.
  arXiv  Detail & Related papers  (2020-12-12T05:30:35Z)
• AvE: Assistance via Empowerment [77.08882807208461]
  We propose a new paradigm for assistance by instead increasing the human's ability to control their environment. This task-agnostic objective preserves the person's autonomy and ability to achieve any eventual state.
  arXiv  Detail & Related papers  (2020-06-26T04:40:11Z)
• Safe Reinforcement Learning via Curriculum Induction [94.67835258431202]
  In safety-critical applications, autonomous agents may need to learn in an environment where mistakes can be very costly. Existing safe reinforcement learning methods make an agent rely on priors that let it avoid dangerous situations. This paper presents an alternative approach inspired by human teaching, where an agent learns under the supervision of an automatic instructor.
  arXiv  Detail & Related papers  (2020-06-22T10:48:17Z)
• Should artificial agents ask for help in human-robot collaborative problem-solving? [0.7251305766151019]
  We propose to start from hypotheses derived from an empirical study in a human-robot interaction. We check whether receiving help from an expert when solving a simple close-ended task allows to accelerate or not the learning of this task. Our experiences have allowed us to conclude that, whether requested or not, a Q-learning algorithm benefits in the same way from expert help as children do.
  arXiv  Detail & Related papers  (2020-05-25T09:15:30Z)
• A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores [85.12096045419686]
  We study the adoption of an algorithmic tool used to assist child maltreatment hotline screening decisions. We first show that humans do alter their behavior when the tool is deployed. We show that humans are less likely to adhere to the machine's recommendation when the score displayed is an incorrect estimate of risk.
  arXiv  Detail & Related papers  (2020-02-19T07:27:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.