Adapting Generalist Robot Policies with Semantic Reinforcement Learning

• URL: http://arxiv.org/abs/2606.31958v1
• Date: Tue, 30 Jun 2026 17:00:33 GMT
• Title: Adapting Generalist Robot Policies with Semantic Reinforcement Learning
• Abstract summary: Generalist robot policies learn a diverse repertoire of behaviors from large-scale pretraining.<n>We propose Semantic Action Reinforcement Learning (SARL) to learn to optimize this prompt space through online interaction.<n>We show SARL unlocks fundamentally new capabilities -- adapting VLA behavior to solve complex, long-horizon tasks.
• Score: 59.009122725071386
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generalist robot policies learn a diverse repertoire of behaviors from large-scale pretraining. In principle, this makes them excellent priors for downstream adaptation via reinforcement learning (RL). In practice, however, standard RL methods leveraging this prior optimize directly over robot actions, requiring the base policy's action distribution to be close to that of a performant policy from the start. This assumption breaks down for complex or long-horizon tasks that fall outside the pretraining distribution. Our key insight is that, for sufficiently expressive generalist policies, language prompts are an effective alternative space for learning to solve such tasks: modulating language inputs elicits skills already within the policy's repertoire, which can be composed to solve tasks beyond its zero-shot capabilities. We propose Semantic Action Reinforcement Learning (SARL), which learns to optimize this prompt space through online interaction, treating the generalist policy as a controllable skill prior. Importantly, leveraging pretrained skills rather than learning new ones from scratch yields structured, semantically meaningful exploration and highly efficient online improvement, and learning to modulate prompts through experience grounds them in induced real-world behaviors for robust task-solving. Across real-world settings and simulated benchmarks, we show SARL unlocks fundamentally new capabilities -- adapting VLA behavior to solve complex, long-horizon tasks -- and significantly outperforms existing approaches for improving robot behavior in deployment.

Related papers

• Improving Robotic Generalist Policies via Flow Reversal Steering [92.97477771370428]
  Generalist policies can learn a wide range of skills from diverse robot datasets.<n>In order to solve or improve on challenging new tasks, we need a way to infer and invoke the appropriate actions from the policy's rich behavioral prior.<n>We propose Flow Reversal Steering (FRS): a method that takes suboptimal but reasonable'' actions, finds their latent noises by passing them through the flow policy in reverse, and maps them to nearby generalist action modes.
  arXiv  Detail & Related papers  (2026-06-11T17:59:45Z)
• PRISM: Personalized Refinement of Imitation Skills for Manipulation via Human Instructions [0.6299766708197883]
  This paper presents PRISM: an instruction-conditioned refinement method for imitation policies in robotic manipulation.<n>Approach bridges Imitation Learning (IL) and Reinforcement Learning (RL) frameworks into a seamless pipeline.<n>Results for a pick-and-place task in a simulated scenario show that proposed method outperforms policies without human feedback.
  arXiv  Detail & Related papers  (2026-03-05T17:05:08Z)
• On-the-Fly VLA Adaptation via Test-Time Reinforcement Learning [61.38273866396522]
  Vision-Language-Action models have emerged as a powerful paradigm for general-purpose robot learning.<n>Current methods remain unsuitable for challenging simulated- or physical-world deployments.<n>We introduce a Test-Time Reinforcement Learning for VLAs framework that enables on-the-fly policy adaptation during inference.
  arXiv  Detail & Related papers  (2026-01-11T01:51:30Z)
• SOP: A Scalable Online Post-Training System for Vision-Language-Action Models [15.86316960521611]
  Vision-language-action (VLA) models achieve strong generalization through large-scale pre-training, but real-world deployment requires expert-level task proficiency in addition to broad generality.<n>Existing post-training approaches for VLA models are typically offline, single-robot, or task-specific.<n>We introduce a Scalable Online Post-training system that enables online, distributed, multi-task post-training of generalist VLA models directly in the physical world.
  arXiv  Detail & Related papers  (2026-01-06T14:25:11Z)
• Agentic Policy Optimization via Instruction-Policy Co-Evolution [44.74237684380034]
  INSPO is a novel framework for instruction-policy co-evolution.<n>It integrates instruction optimization as a dynamic component of the reinforcement learning loop.<n>In experiments, INSPO achieves substantial performance gains with only a marginal increase in computational overhead.
  arXiv  Detail & Related papers  (2025-12-01T17:56:29Z)
• STRAP: Robot Sub-Trajectory Retrieval for Augmented Policy Learning [8.860366821983211]
  STRAP is a technique for leveraging pre-trained vision foundation models and dynamic time warping to retrieve sub-sequences of trajectories from large training corpora in a robust fashion.<n>This work proposes STRAP, a technique for leveraging pre-trained vision foundation models and dynamic time warping to retrieve sub-sequences of trajectories from large training corpora in a robust fashion.
  arXiv  Detail & Related papers  (2024-12-19T18:54:06Z)
• Robotic Control via Embodied Chain-of-Thought Reasoning [86.6680905262442]
  Key limitation of learned robot control policies is their inability to generalize outside their training data.<n>Recent works on vision-language-action models (VLAs) have shown that the use of large, internet pre-trained vision-language models can substantially improve their robustness and generalization ability.<n>We introduce Embodied Chain-of-Thought Reasoning (ECoT) for VLAs, in which we train VLAs to perform multiple steps of reasoning about plans, sub-tasks, motions, and visually grounded features before predicting the robot action.
  arXiv  Detail & Related papers  (2024-07-11T17:31:01Z)
• LGR2: Language Guided Reward Relabeling for Accelerating Hierarchical Reinforcement Learning [23.878883576384002]
  Large language models (LLMs) have shown remarkable abilities in logical reasoning, in-context learning, and code generation.<n>We introduce LGR2, a novel HRL framework that leverages LLMs to generate language-guided reward functions for the higher-level policy.<n>To further enhance sample efficiency in sparse environments, we integrate goal-conditioned hindsight experience relabeling.
  arXiv  Detail & Related papers  (2024-06-09T18:40:24Z)
• PEAR: Primitive Enabled Adaptive Relabeling for Boosting Hierarchical Reinforcement Learning [25.84621883831624]
  Hierarchical reinforcement learning (HRL) has the potential to solve complex long horizon tasks using temporal abstraction and increased exploration.<n>We present primitive enabled adaptive relabeling (PEAR)<n>We first perform adaptive relabeling on a few expert demonstrations to generate efficient subgoal supervision.<n>We then jointly optimize HRL agents by employing reinforcement learning (RL) and imitation learning (IL)
  arXiv  Detail & Related papers  (2023-06-10T09:41:30Z)
• Jump-Start Reinforcement Learning [68.82380421479675]
  We present a meta algorithm that can use offline data, demonstrations, or a pre-existing policy to initialize an RL policy. In particular, we propose Jump-Start Reinforcement Learning (JSRL), an algorithm that employs two policies to solve tasks. We show via experiments that JSRL is able to significantly outperform existing imitation and reinforcement learning algorithms.
  arXiv  Detail & Related papers  (2022-04-05T17:25:22Z)
• Accelerating Robotic Reinforcement Learning via Parameterized Action Primitives [92.0321404272942]
  Reinforcement learning can be used to build general-purpose robotic systems. However, training RL agents to solve robotics tasks still remains challenging. In this work, we manually specify a library of robot action primitives (RAPS), parameterized with arguments that are learned by an RL policy. We find that our simple change to the action interface substantially improves both the learning efficiency and task performance.
  arXiv  Detail & Related papers  (2021-10-28T17:59:30Z)
• Adaptable Automation with Modular Deep Reinforcement Learning and Policy Transfer [8.299945169799795]
  This article develops and tests a Hyper-Actor Soft Actor-Critic (HASAC) RL framework based on the notions of task modularization and transfer learning. The HASAC framework is tested on a new virtual robotic manipulation benchmark, Meta-World. Numerical experiments show superior performance by HASAC over state-of-the-art deep RL algorithms in terms of reward value, success rate, and task completion time.
  arXiv  Detail & Related papers  (2020-11-27T03:09:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.