FlowDAgger: Human-in-the-Loop Adaptation of Generative Robot Policies in Latent Space

• URL: http://arxiv.org/abs/2607.08877v1
• Date: Thu, 09 Jul 2026 19:07:33 GMT
• Title: FlowDAgger: Human-in-the-Loop Adaptation of Generative Robot Policies in Latent Space
• Abstract summary: FlowDAgger is a sample- and compute-efficient method for adapting frozen generative robot policies from human interventions in latent space.<n>Our key idea is action inversion: each human expert action is mapped to the noise that would have produced it under the frozen base policy.<n>We evaluate FlowDAgger in simulation and on real-world bimanual and single-arm manipulation.
• Score: 15.351960740594201
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Pretrained generative robot policies based on flow matching and diffusion have achieved impressive results across a wide range of manipulation tasks. Yet real-world deployments routinely expose failure modes outside the pretraining distribution. Closing these gaps typically requires large-scale data collection or online reinforcement learning on physical hardware, which is impractical for rapid and safe adaptation. We present FlowDAgger, a sample- and compute-efficient method for adapting frozen generative robot policies from human interventions in latent space. Our key idea is action inversion: each human expert action is mapped to the noise that would have produced it under the frozen base policy, using reverse-time integration followed by local refinement. The resulting inverted noise provides supervision for a lightweight latent policy that steers the base model at deployment time, enabling rapid skill acquisition while preserving its behavioral priors. We evaluate FlowDAgger in simulation and on real-world bimanual and single-arm manipulation, adapting both action-head VLAs and world-action models from a handful of interventions. FlowDAgger outperforms supervised fine-tuning and latent-space RL baselines and preserves pretrained skills on held-out tasks, offering a practical path for adapting robot foundation models in the real world. Website: https://microsoft.github.io/FlowDAgger

Related papers

• RoMAN-Flow: Taming Autoregressive Normalizing Flows for Offline Reinforcement Learning in Robotic Manipulation [36.16163751472016]
  offline reinforcement learning framework makes AR-NF policies practical for robotic manipulation.<n>RoMAN-Flow employs a sampling-free, advantage-weighted likelihood objective.<n>Experiments across simulated manipulation benchmarks and real-world robotic platforms demonstrate that RoMAN-Flow achieves competitive policy performance.
  arXiv  Detail & Related papers  (2026-08-20T16:07:56Z)
• HAF: Adapting Generalist VLAs to Humanoid Whole-Body Loco-manipulation via Hierarchical Action Flow and Spectral Latent RL [56.014913786737104]
  We introduce HAF (Humanoid Adaptation Framework) to transfer off-the-shelf generalist VLA foundation models to humanoid whole-body loco-manipulation.<n> HAF-VLA is a hierarchical action-flow generator built on a pretrained flow-matching VLA.<n>HaF-Steer is a latent offline-to-online RL pipeline that leverages flow-matching invertibility and DCT-based dimensionality reduction.
  arXiv  Detail & Related papers  (2026-08-17T17:22:33Z)
• PriGo: Test-Time Primitive Guidance to Diffusion and Flow Policies for Adaptive Robotic Manipulation [28.358317763869664]
  PriGo is a primitive-guided test-time adaptive framework for robust robotic manipulation.<n>We introduce PANet, a lightweight primitive prediction module that infers primitive distributions directly from observations.<n>Unlike prior primitive-conditioned approaches, PriGo operates entirely at test time and can be seamlessly integrated into pretrained diffusion and flow policies.
  arXiv  Detail & Related papers  (2026-07-08T07:08:11Z)
• HiL-ResRL: A Model-Agnostic Finetuning Adapter via Human-in-the-loop Residual Reinforcement Learning [15.872908522637957]
  We introduce a novel, plug-and-play fine-tuning pipeline for Vision-Language-Action (VLA) models.<n>We conceptualize VLA-generated actions as a unified interface, upon which we train a residual policy.<n>Within only 1.5 hour of real-world online RL training, the average success rate exceeds 95% on real robots.
  arXiv  Detail & Related papers  (2026-06-22T05:07:08Z)
• FlowCorrect: Efficient Interactive Correction of Generative Flow Policies for Robotic Manipulation [0.7666240799116112]
  FlowCorrect is a modular interactive imitation learning approach that enables deployment-time adaptation of flow-matching manipulation policies.<n>We evaluate on a real-world robot across four tabletop tasks: pick-and-place, pouring, cup uprighting, and insertion.<n>With a low correction budget, FlowCorrect achieves an 80% success rate on previously failed cases.
  arXiv  Detail & Related papers  (2026-02-25T16:06:49Z)
• RFS: Reinforcement Learning with Residual Flow Steering for Dexterous Manipulation [7.500999283386335]
  Residual Flow Steering (RFS) is a data-efficient reinforcement learning framework for adapting pretrained generative policies.<n>RFS steers a pretrained flow-matching policy by jointly optimizing a residual action and a latent noise distribution.<n>We demonstrate the effectiveness of RFS on dexterous manipulation tasks, showing efficient fine-tuning in both simulation and real-world settings.
  arXiv  Detail & Related papers  (2026-02-02T08:11:57Z)
• Towards Bridging the Gap between Large-Scale Pretraining and Efficient Finetuning for Humanoid Control [18.618486126491334]
  We find that off-policy Soft Actor-Critic (SAC) reliably supports large-scale pretraining of humanoid locomotion policies.<n>SAC-pretrained policies can be finetuned in new environments and out-of-distribution tasks using model-based methods.<n>Overall, the approach couples the wall-clock efficiency of large-scale simulation during pretraining with the sample efficiency of model-based learning during fine-tuning.
  arXiv  Detail & Related papers  (2026-01-29T07:43:24Z)
• SPACeR: Self-Play Anchoring with Centralized Reference Models [50.55045557371374]
  Sim agent policies are realistic, human-like, fast, and scalable in multi-agent settings.<n>Recent progress in imitation learning with large diffusion-based or tokenized models has shown that behaviors can be captured directly from human driving data.<n>We propose SPACeR, a framework that leverages a pretrained tokenized autoregressive motion model as a central reference policy.
  arXiv  Detail & Related papers  (2025-10-20T19:53:02Z)
• Dexplore: Scalable Neural Control for Dexterous Manipulation from Reference-Scoped Exploration [58.4036440289082]
  Hand-object motion-capture (MoCap) offer large-scale, contact-rich demonstrations and hold promise for dexterous robotic scopes.<n>We introduce Dexplore, a unified single-loop optimization that performs repositories and tracking to learn robot control policies directly from MoCap at scale.
  arXiv  Detail & Related papers  (2025-09-11T17:59:07Z)
• Align-Then-stEer: Adapting the Vision-Language Action Models through Unified Latent Guidance [63.33213516925946]
  We introduce textbfAlign-Then-stEer (textttATE), a novel, data-efficient, and plug-and-play adaptation framework.<n>Our work presents a general and lightweight solution that greatly enhances the practicality of deploying VLA models to new robotic platforms and tasks.
  arXiv  Detail & Related papers  (2025-09-02T07:51:59Z)
• Action Flow Matching for Continual Robot Learning [54.10050120844738]
  Continual learning in robotics seeks systems that can constantly adapt to changing environments and tasks.<n>We introduce a generative framework leveraging flow matching for online robot dynamics model alignment.<n>We find that by transforming the actions themselves rather than exploring with a misaligned model, the robot collects informative data more efficiently.
  arXiv  Detail & Related papers  (2025-04-25T16:26:15Z)
• Let Offline RL Flow: Training Conservative Agents in the Latent Space of Normalizing Flows [58.762959061522736]
  offline reinforcement learning aims to train a policy on a pre-recorded and fixed dataset without any additional environment interactions. We build upon recent works on learning policies in latent action spaces and use a special form of Normalizing Flows for constructing a generative model. We evaluate our method on various locomotion and navigation tasks, demonstrating that our approach outperforms recently proposed algorithms.
  arXiv  Detail & Related papers  (2022-11-20T21:57:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.