GAINS: Leveraging Inconsistent Human Intervention Signals in Reinforcement Learning

• URL: http://arxiv.org/abs/2608.15707v1
• Date: Sun, 16 Aug 2026 12:19:04 GMT
• Title: GAINS: Leveraging Inconsistent Human Intervention Signals in Reinforcement Learning
• Abstract summary: We present a framework for leveraging inconsistent human intervention signals in reinforcement learning (RL)<n>We employ distributional RL with quantile Q-networks to model the return variability induced by sparse task rewards and inconsistent human interventions.<n>We evaluate GAINS on four diverse simulated manipulation tasks and two challenging real-world scenarios against state-of-the-art intervention-based methods.
• Score: 35.461668686434244
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Correcting robot manipulation policies through human intervention holds great promise for real-world deployment, yet human operators are inherently imperfect in both the actions they provide and the timing of their intervention signals. While the former has been extensively discussed in reinforcement learning (RL), the latter remains underexplored. At high control frequencies, human intervention signals are often delayed and inconsistent across time and state space. In this work, we present GAINS, a framework for leveraging inconsistent human intervention signals in RL. At the core of GAINS, we employ distributional RL with quantile Q-networks to model the return variability induced by sparse task rewards and inconsistent human interventions. Building on this distributional representation, we introduce a pessimistic exploration strategy that promotes safe and sample-efficient learning under human corrections. We evaluate GAINS on four diverse simulated manipulation tasks and two challenging real-world scenarios against state-of-the-art intervention-based methods. GAINS achieves a 22% higher task success rate than RLIF and improves recovery success by up to 43% in failure scenarios. These results highlight the importance of modeling return variability induced by human imperfection for real-world deployment of intervention-based learning.

Related papers

• How to Learn from What a Human Would Avoid? Intervention-Aware World Models with Real-World RL for Dexterous Manipulation [27.05205948648607]
  We present WHIRL, a safety-aware RL framework that transforms binary human interventions into forward-predictive signals for proactive risk avoidance.<n>We evaluate our framework on a 16-DoF LEAP Hand across tasks spanning convex and irregular object grasping, prismatic manipulation, and long-horizon multi-stage tasks.
  arXiv  Detail & Related papers  (2026-09-05T10:28:23Z)
• ROVE: Unlocking Human Interventions for Humanoid Manipulation via Reinforcement Learning [44.43333438148224]
  We propose ROVE, a reinforcement learning framework for humanoid VLA post-training with imperfect human interventions.<n>First, ROVE introduces a human-in-the-loop pipeline capable of collecting deployment and intervention data for humanoid manipulation.<n>Second, it utilizes Optimistic Value Estimation (OVE) to prioritize high-value behaviors from mixed-quality trajectories.
  arXiv  Detail & Related papers  (2026-06-15T17:45:06Z)
• UniIntervene: Agentic Intervention for Efficient Real-World Reinforcement Learning [10.315300563393782]
  We propose UniIntervene, an agentic intervention model that detects unproductive exploration and autonomously recovers the policy toward high-value states.<n>Experiments on diverse real-world manipulation tasks demonstrate that UniIntervene improves the average success rate by 8.6% while reducing human interventions by 57% relative to state-of-the-art HiL-RL baselines.
  arXiv  Detail & Related papers  (2026-06-10T17:38:24Z)
• OHP-RL: Online Human Preference as Guidance in Reinforcement Learning for Robot Manipulation [16.28822074948203]
  Online Human Preference as Guidance in Reinforcement Learning (OHP-RL) is a framework that leverages human interventions as preference information to guide policy learning.<n>OHP-RL consistently achieves strong success rates, faster convergence, and substantially lower human intervention effort than prior approaches.
  arXiv  Detail & Related papers  (2026-05-15T14:02:34Z)
• Human-in-the-loop Online Rejection Sampling for Robotic Manipulation [55.99788088622936]
  Hi-ORS stabilizes value estimation by filtering out negatively rewarded samples during online fine-tuning.<n>Hi-ORS fine-tunes a pi-base policy to master contact-rich manipulation in just 1.5 hours of real-world training.
  arXiv  Detail & Related papers  (2025-10-30T11:53:08Z)
• MEReQ: Max-Ent Residual-Q Inverse RL for Sample-Efficient Alignment from Intervention [76.83428371942735]
  We introduce MEReQ (Maximum-Entropy Residual-Q Inverse Reinforcement Learning), designed for sample-efficient alignment from human intervention.<n>MereQ infers a residual reward function that captures the discrepancy between the human expert's and the prior policy's underlying reward functions.<n>It then employs Residual Q-Learning (RQL) to align the policy with human preferences using this residual reward function.
  arXiv  Detail & Related papers  (2024-06-24T01:51:09Z)
• Safety-compliant Generative Adversarial Networks for Human Trajectory Forecasting [95.82600221180415]
  Human forecasting in crowds presents the challenges of modelling social interactions and outputting collision-free multimodal distribution. We introduce SGANv2, an improved safety-compliant SGAN architecture equipped with motion-temporal interaction modelling and a transformer-based discriminator design.
  arXiv  Detail & Related papers  (2022-09-25T15:18:56Z)
• Persistent Reinforcement Learning via Subgoal Curricula [114.83989499740193]
  Value-accelerated Persistent Reinforcement Learning (VaPRL) generates a curriculum of initial states. VaPRL reduces the interventions required by three orders of magnitude compared to episodic reinforcement learning.
  arXiv  Detail & Related papers  (2021-07-27T16:39:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.