EvoHIL: Self-Evolving Reward and Flow-Matched Policy Optimization for Robust Human-in-the-Loop Reinforcement Learning

• URL: http://arxiv.org/abs/2608.03872v1
• Date: Tue, 04 Aug 2026 16:12:05 GMT
• Title: EvoHIL: Self-Evolving Reward and Flow-Matched Policy Optimization for Robust Human-in-the-Loop Reinforcement Learning
• Abstract summary: EvoHIL is a unified framework that adapts the reward model, action generator, and visual do main within a staged human-in-the-loop learning process.<n>It improves task success, agreement with human-confirmation labels, motion smoothness, and completion time relative to human-in-the-loop and imitation baselines.
• Score: 14.891400906901433
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Human-in-the-loop reinforcement learning (HIL-RL) enables robots to learn contact-rich manipulation from limited real-world interaction, but deployment exposes three coupled limitations: static visual reward models fail under scene changes; independently sampled actions cause temporally inconsistent motion; and vision-based policies remain sensitive to appearance shifts. We present EvoHIL, a unified framework that adapts the reward model, action generator, and visual do main within a staged human-in-the-loop learning process. First, self-evolving reward (SER) adapts the success classifier from human-confirmed positives and provisional weak negatives. Second, Action Flow Stabilization (AFS) generates temporally coherent action chunks through flow matching, grounding policy updates in executed action prefixes and demonstrated behavior. Third, retention-aware offline fine-tuning replays relit interaction data while anchoring the AFS actor-critic to prior behavior, adapting the visual domain without additional robot interaction. Across six manipulation tasks on Franka FR3 and SO-101 arms under a controlled lighting shift, EvoHIL improves task success, agreement with human-confirmation labels, motion smoothness, and completion time relative to human-in-the-loop and imitation baselines.Project page: https://anonymous4366.github.io/EvoHIL/

Related papers

• Unifying Physics-Based Humanoid Interaction with a Context-Conditioned Interaction Prior [20.58647951068921]
  CHIP is a physics-grounded framework for learning reusable humanoid interaction skills from heterogeneous motion data.<n>We first learn physics-based motion-imitation policies that acquire grounded teacher behaviors from heterogeneous interaction data.<n>We then distill these behaviors into a context-conditioned interaction prior that captures reusable motion structure across locomotion and manipulation.
  arXiv  Detail & Related papers  (2026-09-06T13:16:48Z)
• HAF: Adapting Generalist VLAs to Humanoid Whole-Body Loco-manipulation via Hierarchical Action Flow and Spectral Latent RL [56.014913786737104]
  We introduce HAF (Humanoid Adaptation Framework) to transfer off-the-shelf generalist VLA foundation models to humanoid whole-body loco-manipulation.<n> HAF-VLA is a hierarchical action-flow generator built on a pretrained flow-matching VLA.<n>HaF-Steer is a latent offline-to-online RL pipeline that leverages flow-matching invertibility and DCT-based dimensionality reduction.
  arXiv  Detail & Related papers  (2026-08-17T17:22:33Z)
• Skill Self-Play: Pushing the Frontier of LLM Capability with Co-Evolving Skills [53.15216425100295]
  We introduce Skill Self-Play (Skill-SP), a co-evolutionary framework comprising a proposer, a solver, and a skill controller.<n>The proposer generates challenging tasks conditioned on dynamically sampled skills; the solver explores candidate solutions to push its capability boundaries; and the skill controller collects execution feedback to update and expand the skill library.<n>This interactive co-evolution effectively bridges the gap between structured verification and open-ended exploration.
  arXiv  Detail & Related papers  (2026-07-24T17:59:22Z)
• CausalDrive: Real-time Causal World Models for Autonomous Driving [60.66609721457312]
  We present CausalDrive, a controllable, real-time foundation driving world.<n>CaulDrive operates solely on the initial front-view frame, the ego-vehicle's trajectory, and a macroscopic text prompt.<n>We demonstrate its versatility across three downstream applications: (1) generative closed-loop evaluation with significantly mitigated collision artifacts, (2) large-scale Reinforcement Learning (RL) post-training driven by a Video2Reward module, and (3) real-time human-in-the-loop simulation.
  arXiv  Detail & Related papers  (2026-06-13T15:04:44Z)
• HOIST: Humanoid Optimization with Imitation and Sample-efficient Tuning for Manipulating Suspended Loads [26.63242922377076]
  We present HOIST-Humanoid optimized with Imitation and Sample-efficient Tuning for manipulating suspended loads.<n> HOIST first finetunes a high-level vision--action (VLA) policy from virtual-reality (VR) teleoperation demonstrations.<n>It then uses VLA rollouts and iterative batched RL to improve placement accuracy and stopping behavior.
  arXiv  Detail & Related papers  (2026-05-29T18:33:32Z)
• From Imagined Futures to Executable Actions: Mixture of Latent Actions for Robot Manipulation [88.39072412680633]
  We propose MoLA, a control-oriented interface that transforms imagined future videos into executable representations.<n>We evaluate our approach on simulated benchmarks and real-world robot manipulation tasks.
  arXiv  Detail & Related papers  (2026-05-12T14:15:16Z)
• LatentPilot: Scene-Aware Vision-and-Language Navigation by Dreaming Ahead with Latent Visual Reasoning [51.969318585152116]
  LatentPilot exploits future observations during training as a valuable data source to learn action-conditioned visual dynamics.<n>We propose a flywheel-style training mechanism that iteratively collects on-policy trajectories and retrains the model to better match the agent's behavior distribution.<n> Experiments on R2R-CE, RxR-CE, and R2R-PE benchmarks achieve new SOTA results, and real-robot tests across diverse environments demonstrate LatentPilot's superior understanding of environment-action dynamics in scene.
  arXiv  Detail & Related papers  (2026-03-31T02:21:59Z)
• Self-Correcting VLA: Online Action Refinement via Sparse World Imagination [55.982504915794514]
  We propose Self-Correcting VLA (SC-VLA), which achieve self-improvement by intrinsically guiding action refinement through sparse imagination.<n>SC-VLA achieve state-of-the-art performance, yielding the highest task throughput with 16% fewer steps and a 9% higher success rate than the best-performing baselines.
  arXiv  Detail & Related papers  (2026-02-25T06:58:06Z)
• FlowHOI: Flow-based Semantics-Grounded Generation of Hand-Object Interactions for Dexterous Robot Manipulation [23.19464039872024]
  FlowHOI is a flow-matching framework that generates temporally coherent HOI sequences.<n>We show that FlowHOI achieves the highest action recognition accuracy and a 1.7$times$ higher physics simulation success rate.
  arXiv  Detail & Related papers  (2026-02-13T20:46:08Z)
• ConLA: Contrastive Latent Action Learning from Human Videos for Robotic Manipulation [27.54751123419347]
  We propose ConLA, an unsupervised pretraining framework for learning robotic policies from human videos.<n>By pretraining solely on human videos, our method for the first time surpasses the performance obtained with real robot trajectory pretraining.
  arXiv  Detail & Related papers  (2026-01-31T06:40:57Z)
• Learning Whole-Body Human-Humanoid Interaction from Human-Human Demonstrations [63.80827184637476]
  We introduce D-STAR, a hierarchical policy that disentangles when to act from where to act.<n>We validate our framework through extensive and rigorous simulations.
  arXiv  Detail & Related papers  (2026-01-14T14:37:06Z)
• Bootstrap Dynamic-Aware 3D Visual Representation for Scalable Robot Learning [20.535171893983794]
  We introduce AFRO, a self-supervised framework that learns 3D representations without action or reconstruction supervision.<n>We employ feature differencing and inverse-consistency supervision, improving the quality and stability of visual features.<n>When combined with Diffusion Policy, AFRO substantially increases manipulation success rates across 16 simulated and 4 real-world tasks.
  arXiv  Detail & Related papers  (2025-11-25T02:15:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.