Robot-Gated Interactive Imitation Learning with Adaptive Intervention Mechanism

• URL: http://arxiv.org/abs/2506.09176v1
• Date: Tue, 10 Jun 2025 18:43:26 GMT
• Title: Robot-Gated Interactive Imitation Learning with Adaptive Intervention Mechanism
• Authors: Haoyuan Cai, Zhenghao Peng, Bolei Zhou,
• Abstract summary: Interactive Imitation Learning (IIL) allows agents to acquire desired behaviors through human interventions.<n>We propose the Adaptive Intervention Mechanism (AIM), a novel robot-gated IIL algorithm that learns an adaptive criterion for requesting human demonstrations.
• Score: 48.41735416075536
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Interactive Imitation Learning (IIL) allows agents to acquire desired behaviors through human interventions, but current methods impose high cognitive demands on human supervisors. We propose the Adaptive Intervention Mechanism (AIM), a novel robot-gated IIL algorithm that learns an adaptive criterion for requesting human demonstrations. AIM utilizes a proxy Q-function to mimic the human intervention rule and adjusts intervention requests based on the alignment between agent and human actions. By assigning high Q-values when the agent deviates from the expert and decreasing these values as the agent becomes proficient, the proxy Q-function enables the agent to assess the real-time alignment with the expert and request assistance when needed. Our expert-in-the-loop experiments reveal that AIM significantly reduces expert monitoring efforts in both continuous and discrete control tasks. Compared to the uncertainty-based baseline Thrifty-DAgger, our method achieves a 40% improvement in terms of human take-over cost and learning efficiency. Furthermore, AIM effectively identifies safety-critical states for expert assistance, thereby collecting higher-quality expert demonstrations and reducing overall expert data and environment interactions needed. Code and demo video are available at https://github.com/metadriverse/AIM.

Related papers

• Exploring Big Five Personality and AI Capability Effects in LLM-Simulated Negotiation Dialogues [16.07828032939124]
  This paper presents an evaluation framework for agentic AI systems in mission-critical negotiation contexts.<n>Using Sotopia as a simulation testbed, we present two experiments that systematically evaluated how personality traits and AI agent characteristics influence social negotiation outcomes.
  arXiv  Detail & Related papers  (2025-06-19T00:14:56Z)
• Interactive Double Deep Q-network: Integrating Human Interventions and Evaluative Predictions in Reinforcement Learning of Autonomous Driving [16.379623042562415]
  This study introduces Interactive Double Deep Q-network (iDDQN), a Human-in-the-Loop (HITL) approach that enhances Reinforcement Learning (RL)<n>Our proposed iDDQN method modifies the Q-value update equation to integrate human and agent actions, establishing a collaborative approach for policy development.<n> Empirical results in simulated autonomous driving scenarios demonstrate that iDDQN outperforms established approaches.
  arXiv  Detail & Related papers  (2025-04-28T05:25:18Z)
• VeriLA: A Human-Centered Evaluation Framework for Interpretable Verification of LLM Agent Failures [3.075266204492352]
  Large language model (LLM) agents in compound AI systems often fail to meet human standards, leading to errors that compromise the system's overall performance.<n>This paper introduces a human-centered evaluation framework for verifying LLM Agent failures (VeriLA)<n>VeriLA systematically assesses agent failures to reduce human effort and make these agent failures interpretable to humans.
  arXiv  Detail & Related papers  (2025-03-16T21:11:18Z)
• AgentDAM: Privacy Leakage Evaluation for Autonomous Web Agents [75.85554113398626]
  We introduce a new benchmark AgentDAM that measures if AI web-navigation agents follow the privacy principle of data minimization''<n>Our benchmark simulates realistic web interaction scenarios end-to-end and is adaptable to all existing web navigation agents.
  arXiv  Detail & Related papers  (2025-03-12T19:30:31Z)
• Interactive Agents to Overcome Ambiguity in Software Engineering [61.40183840499932]
  AI agents are increasingly being deployed to automate tasks, often based on ambiguous and underspecified user instructions.<n>Making unwarranted assumptions and failing to ask clarifying questions can lead to suboptimal outcomes.<n>We study the ability of LLM agents to handle ambiguous instructions in interactive code generation settings by evaluating proprietary and open-weight models on their performance.
  arXiv  Detail & Related papers  (2025-02-18T17:12:26Z)
• MEReQ: Max-Ent Residual-Q Inverse RL for Sample-Efficient Alignment from Intervention [81.56607128684723]
  We introduce MEReQ (Maximum-Entropy Residual-Q Inverse Reinforcement Learning), designed for sample-efficient alignment from human intervention. MereQ infers a residual reward function that captures the discrepancy between the human expert's and the prior policy's underlying reward functions. It then employs Residual Q-Learning (RQL) to align the policy with human preferences using this residual reward function.
  arXiv  Detail & Related papers  (2024-06-24T01:51:09Z)
• Reinforcement Learning Interventions on Boundedly Rational Human Agents in Frictionful Tasks [25.507656595628376]
  We introduce a framework in which an AI agent intervenes on the parameters of a Markov Decision Process (MDP) belonging to a boundedly rational human agent. We show that AI planning with our human models can lead to helpful policies on a wide range of more complex, ground-truth humans.
  arXiv  Detail & Related papers  (2024-01-26T14:59:48Z)
• Online Decision Mediation [72.80902932543474]
  Consider learning a decision support assistant to serve as an intermediary between (oracle) expert behavior and (imperfect) human behavior. In clinical diagnosis, fully-autonomous machine behavior is often beyond ethical affordances.
  arXiv  Detail & Related papers  (2023-10-28T05:59:43Z)
• A Cognitive Framework for Delegation Between Error-Prone AI and Human Agents [0.0]
  We investigate the use of cognitively inspired models of behavior to predict the behavior of both human and AI agents. The predicted behavior is used to delegate control between humans and AI agents through the use of an intermediary entity.
  arXiv  Detail & Related papers  (2022-04-06T15:15:21Z)
• ThriftyDAgger: Budget-Aware Novelty and Risk Gating for Interactive Imitation Learning [23.177329496817105]
  ThriftyDAgger is an algorithm for querying a human supervisor given a desired budget of human interventions. Experiments suggest that ThriftyDAgger's intervention criteria balances task performance and supervisor burden more effectively than prior algorithms.
  arXiv  Detail & Related papers  (2021-09-17T01:21:16Z)
• Watch-And-Help: A Challenge for Social Perception and Human-AI Collaboration [116.28433607265573]
  We introduce Watch-And-Help (WAH), a challenge for testing social intelligence in AI agents. In WAH, an AI agent needs to help a human-like agent perform a complex household task efficiently. We build VirtualHome-Social, a multi-agent household environment, and provide a benchmark including both planning and learning based baselines.
  arXiv  Detail & Related papers  (2020-10-19T21:48:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.