Unified Noise Steering for Efficient Human-Guided VLA Adaptation

• URL: http://arxiv.org/abs/2605.10821v1
• Date: Mon, 11 May 2026 16:37:34 GMT
• Title: Unified Noise Steering for Efficient Human-Guided VLA Adaptation
• Abstract summary: Diffusion-based vision-language-action (VLA) models have emerged as strong priors for robotic manipulation.<n>Noise-space RL lowers the cost by keeping the pretrained VLA fixed as a denoising generator while updating only a lightweight actor that predicts the noise.<n>We propose UniSteer, a Unified Noise Steering framework that combines human corrective guidance with noise-space RL through approximate action-to-noise inversion.
• Score: 28.625432744456763
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Diffusion-based vision-language-action (VLA) models have emerged as strong priors for robotic manipulation, yet adapting them to real-world distributions remains challenging. In particular, on-robot reinforcement learning (RL) is expensive and time-consuming, so effective adaptation depends on efficient policy improvement within a limited budget of real-world interactions. Noise-space RL lowers the cost by keeping the pretrained VLA fixed as a denoising generator while updating only a lightweight actor that predicts the noise. However, its performance is still limited due to inefficient autonomous exploration. Human corrective interventions can reduce this exploration burden, but they are naturally provided in action space, whereas noise-space finetuning requires supervision over noise variables. To address these challenges, we propose UniSteer, a Unified Noise Steering framework that combines human corrective guidance with noise-space RL through approximate action-to-noise inversion. Given a human corrective action, UniSteer inverts the frozen flow-matching decoder to recover a noise target, which provides supervised guidance for the same noise actor that is simultaneously optimized via reinforcement learning. Real-world experiments on diverse manipulation tasks show that UniSteer adapts more efficiently than strong noise-space RL and action-space human-in-the-loop baselines, improving the success rate from 20% to 90% in 66 minutes on average across four real-world adaptation tasks.

Related papers

• HAF: Adapting Generalist VLAs to Humanoid Whole-Body Loco-manipulation via Hierarchical Action Flow and Spectral Latent RL [56.014913786737104]
  We introduce HAF (Humanoid Adaptation Framework) to transfer off-the-shelf generalist VLA foundation models to humanoid whole-body loco-manipulation.<n> HAF-VLA is a hierarchical action-flow generator built on a pretrained flow-matching VLA.<n>HaF-Steer is a latent offline-to-online RL pipeline that leverages flow-matching invertibility and DCT-based dimensionality reduction.
  arXiv  Detail & Related papers  (2026-08-17T17:22:33Z)
• StructRL: Structured Action-Space Exploration for Flow-Based VLAs [81.43458629514139]
  Flow-based Vision-Language-Action models are now widely used for continuous robotic manipulation.<n>Online reinforcement learning is emerging as a key technique for adapting them to new tasks.<n>We show that effective robot exploration calls for structured noise: temporally smooth and scaled differently across action groups.
  arXiv  Detail & Related papers  (2026-08-15T09:31:29Z)
• Global Prior Meets Local Consistency: Dual-Memory Augmented Vision-Language-Action Model for Efficient Robotic Manipulation [95.89924101984566]
  We introduce OptimusVLA, a dual-memory VLA framework with Global Prior Memory (GPM) and Local Consistency Memory (LCM)<n>GPM replaces Gaussian noise with task-level priors retrieved from semantically similar trajectories.<n>LCM injects a learned consistency constraint that enforces temporal coherence and smoothness of trajectory.
  arXiv  Detail & Related papers  (2026-02-22T15:39:34Z)
• RFS: Reinforcement Learning with Residual Flow Steering for Dexterous Manipulation [7.500999283386335]
  Residual Flow Steering (RFS) is a data-efficient reinforcement learning framework for adapting pretrained generative policies.<n>RFS steers a pretrained flow-matching policy by jointly optimizing a residual action and a latent noise distribution.<n>We demonstrate the effectiveness of RFS on dexterous manipulation tasks, showing efficient fine-tuning in both simulation and real-world settings.
  arXiv  Detail & Related papers  (2026-02-02T08:11:57Z)
• Learning to Explore with Parameter-Space Noise: A Deep Dive into Parameter-Space Noise for Reinforcement Learning with Verifiable Rewards [16.22162269278471]
  PSN-RLVR perturbs policy parameters before rollout generation to induce temporally consistent, trajectory-level exploration.<n>We propose a computationally efficient real-time adaptive noise scheduler driven by a lightweight surrogate that combines semantic diversity with normalized self-certainty.
  arXiv  Detail & Related papers  (2026-01-30T13:10:30Z)
• Towards Bridging the Gap between Large-Scale Pretraining and Efficient Finetuning for Humanoid Control [18.618486126491334]
  We find that off-policy Soft Actor-Critic (SAC) reliably supports large-scale pretraining of humanoid locomotion policies.<n>SAC-pretrained policies can be finetuned in new environments and out-of-distribution tasks using model-based methods.<n>Overall, the approach couples the wall-clock efficiency of large-scale simulation during pretraining with the sample efficiency of model-based learning during fine-tuning.
  arXiv  Detail & Related papers  (2026-01-29T07:43:24Z)
• Real-world Reinforcement Learning from Suboptimal Interventions [39.23110010675281]
  SiLRI is a state-wise Lagrangian reinforcement learning algorithm for real-world robot manipulation tasks.<n>Built upon a human-as-copilot teleoperation system, our algorithm is evaluated through real-world experiments on diverse manipulation tasks.
  arXiv  Detail & Related papers  (2025-12-30T15:26:42Z)
• ENJ: Optimizing Noise with Genetic Algorithms to Jailbreak LSMs [61.09812971042288]
  Evolutionary Noise Jailbreak (ENJ)<n>This paper proposes a genetic algorithm to transform environmental noise from a passive interference into an actively optimizable attack carrier for jailbreaking LSMs.<n>Experiments on multiple mainstream speech models show that ENJ's attack effectiveness is significantly superior to existing baseline methods.
  arXiv  Detail & Related papers  (2025-09-14T06:39:38Z)
• QuietPaw: Learning Quadrupedal Locomotion with Versatile Noise Preference Alignment [32.730540558720314]
  Noise from quadrupedal robots can disrupt human-centered environments like homes, offices, and hospitals.<n>We propose QuietPaw, a constrained learning-based algorithm that enables flexible, noise-aware locomotion by conditioning policy behavior on noise-reduction levels.<n>We validate our approach in simulation and the real world, demonstrating that CNCP can effectively balance locomotion performance and noise constraints.
  arXiv  Detail & Related papers  (2025-03-06T23:20:04Z)
• Latent Exploration for Reinforcement Learning [87.42776741119653]
  In Reinforcement Learning, agents learn policies by exploring and interacting with the environment. We propose LATent TIme-Correlated Exploration (Lattice), a method to inject temporally-correlated noise into the latent state of the policy network.
  arXiv  Detail & Related papers  (2023-05-31T17:40:43Z)
• A noise-robust acoustic method for recognizing foraging activities of grazing cattle [35.21388806827219]
  We present the operating principle and generalization capability of an acoustic method called Noise-Robust Foraging Activity Recognizer (NRFAR) In noiseless conditions, NRFAR reached an average balanced accuracy of 86.4%, outperforming two previous acoustic methods by more than 7.5%. NRFAR has been shown to be effective in harsh free-ranging environments and could be used as a reliable solution to improve pasture management and monitor the health and welfare of dairy cows.
  arXiv  Detail & Related papers  (2023-04-28T13:06:14Z)
• Improve Noise Tolerance of Robust Loss via Noise-Awareness [60.34670515595074]
  We propose a meta-learning method which is capable of adaptively learning a hyper parameter prediction function, called Noise-Aware-Robust-Loss-Adjuster (NARL-Adjuster for brevity) Four SOTA robust loss functions are attempted to be integrated with our algorithm, and comprehensive experiments substantiate the general availability and effectiveness of the proposed method in both its noise tolerance and performance.
  arXiv  Detail & Related papers  (2023-01-18T04:54:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.