LabVLA: Grounding Vision-Language-Action Models in Scientific Laboratories

• URL: http://arxiv.org/abs/2606.13578v2
• Date: Mon, 15 Jun 2026 17:50:36 GMT
• Title: LabVLA: Grounding Vision-Language-Action Models in Scientific Laboratories
• Abstract summary: Vision-Language-Action (VLA) models provide one possible interface between written protocols and robot execution.<n>Existing policies are trained mostly on household and tabletop demonstrations and rarely encounter the instruments, transparent liquids, or fixed protocol found in scientific laboratories.<n>We present LabVLA, trained with a two-stage recipe: FAST action token pretraining first makes the Qwen3-VL-4B-Instruct backbone action aware before any continuous control is learned, and flow matching posttraining then attaches a DiT action expert under knowledge insulation.
• Score: 95.20367571157679
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Scientific laboratories increasingly rely on AI systems to reason about experiments, but the physical act of doing science remains largely outside their reach. AI can help read literature, generate hypotheses, and plan protocols, yet the execution of those protocols at the bench still requires a human operator. Vision-Language-Action (VLA) models provide one possible interface between written protocols and robot execution, but existing policies are trained mostly on household and tabletop demonstrations and rarely encounter the instruments, transparent liquids, or fixed protocol workflows found in scientific laboratories. Closing this gap requires both laboratory-specific supervision and a unified learning framework that can accommodate the diverse robot embodiments used to execute experimental protocols. We therefore identify data and embodiment as central bottlenecks alongside model design. To address the data side, we build RoboGenesis, a simulation-based workflow and data engine that composes configured laboratory workflows from atomic skills, validates and filters rollouts, and exports structured demonstrations across supported robot profiles. On the policy side, we present LabVLA, trained with a two-stage recipe: FAST action token pretraining first makes the Qwen3-VL-4B-Instruct backbone action aware before any continuous control is learned, and flow matching posttraining then attaches a DiT action expert under knowledge insulation. On the LabUtopia benchmark, LabVLA achieves the highest average success rate among all evaluated baselines under both in-distribution and out-of-distribution settings.

Related papers

• LabDex: A Hierarchical Benchmark for Dexterous Manipulation in Laboratories [40.311566677309536]
  LabDex is a large-scale real-world dataset and benchmark for dexterous manipulation in chemistry laboratories.<n>It is organized around a hierarchical task taxonomy spanning atomic skills, compositional tasks, and long-horizon experiments.<n>We conduct cross-level evaluations of representative robot learning methods in both real-world and simulation environments.
  arXiv  Detail & Related papers  (2026-08-19T07:11:06Z)
• Labimus: A Simulation and Benchmark for Humanoid Dexterous Manipulation in Chemical Laboratory [47.77200587195896]
  Labimus is the first benchmark for humanoid dexterous manipulation in organic chemistry laboratories.<n>It reconstructs over 30 functionally faithful assets from real organic chemistry workstations through real-to-sim modeling.<n>We introduce a precision-aware evaluation protocol designed to jointly measure task completion, experimental precision, and long-horizon execution.
  arXiv  Detail & Related papers  (2026-06-30T02:14:28Z)
• Claw AI Lab: An Autonomous Multi-Agent Research Team [73.07229229907435]
  We present Claw AI Lab, a lab-native autonomous research platform.<n>Rather than centering the system around a single agent or a fixed serial workflow, we allow users to instantiate a full research team from one prompt.<n>Claw-Code Harness connects locals, datasets, and checkpoints to runnable experiments and feeds execution artifacts back into the research loop.
  arXiv  Detail & Related papers  (2026-05-21T16:02:53Z)
• From Prompts to Protocols: An AI Agent for Laboratory Automation [1.9609766210591626]
  We present an AI agent architecture that integrates large language models with laboratory orchestration.<n>The AI agent operates under an agentic loop with automated validation and error correction.<n>It supports the complete experimental lifecycle: creating protocols, running and monitoring both protocols and closed-loop optimization campaigns.
  arXiv  Detail & Related papers  (2026-05-15T18:52:41Z)
• BioProVLA-Agent: An Affordable, Protocol-Driven, Vision-Enhanced VLA-Enabled Embodied Multi-Agent System with Closed-Loop-Capable Reasoning for Biological Laboratory Manipulation [15.461936066333633]
  BioProVLA-Agent is a protocol-driven, vision-enhanced embodied multi-agent system for biological manipulation.<n>It integrates protocol parsing, visual state verification, and embodied execution in a closed-loop workflow.<n>We evaluate the system on a hierarchical benchmark covering 15 atomic tasks, 6 composite tasks, and 3 bimanual tasks.
  arXiv  Detail & Related papers  (2026-05-08T06:15:40Z)
• LabBuilder: Protocol-Grounded 3D Layout Generation for Interactable and Safe Laboratory [93.63620478256063]
  We present LabBuilder, an end-to-end system that generates and verifies 3D laboratory layouts from concise textual specifications.<n>LabBuilder significantly outperforms existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-05-04T07:24:56Z)
• Sci-VLA: Agentic VLA Inference Plugin for Long-Horizon Tasks in Scientific Experiments [49.02509634515056]
  Recent vision-language-action models offer a promising foundation for robotic laboratories.<n>Experiments typically involve long-horizon tasks composed of multiple atomic tasks.<n>While VLA models fine-tuned for scientific tasks can reliably execute atomic experimental actions, they often fail to perform composite tasks formed by reordering and composing these known atomic actions.
  arXiv  Detail & Related papers  (2026-02-10T05:50:19Z)
• PRISM: Protocol Refinement through Intelligent Simulation Modeling [4.839327116611717]
  We introduce PRISM, a framework that automates the design, validation, and execution of experimental protocols.<n>PRISM uses a set of language-model-based agents that work together to generate and refine experimental steps.<n>We demonstrate PRISM as a practical end-to-end workflow that bridges language-based protocol generation, simulation-based validation, and automated robotic execution.
  arXiv  Detail & Related papers  (2026-01-08T20:15:28Z)
• Bohrium + SciMaster: Building the Infrastructure and Ecosystem for Agentic Science at Scale [82.20980951765891]
  We argue that scaling agentic science requires an infrastructure-and-ecosystem approach, instantiated Bohrium+SciMaster.<n>Bohrium acts as a managed, traceable hub for AI4S assets that turns diverse scientific data, software, compute, and laboratory systems into agent-ready capabilities.<n>SciMaster orchestrates these capabilities into long-horizon scientific, on which scientific agents can be composed and executed.
  arXiv  Detail & Related papers  (2025-12-23T16:04:41Z)
• LabUtopia: High-Fidelity Simulation and Hierarchical Benchmark for Scientific Embodied Agents [103.65422553044816]
  LabUtopia is a comprehensive simulation and benchmarking suite designed to facilitate the development of generalizable, reasoning-capable embodied agents.<n>It supports 30 distinct tasks and includes more than 200 scene and instrument assets.<n>We demonstrate that LabUtopia offers a powerful platform for advancing the integration of perception, planning, and control in scientific-purpose agents.
  arXiv  Detail & Related papers  (2025-05-28T17:50:53Z)
• AlabOS: A Python-based Reconfigurable Workflow Management Framework for Autonomous Laboratories [3.8330070166920556]
  We introduce AlabOS, a general-purpose software framework for orchestrating experiments and managing resources. AlabOS features a reconfigurable experiment workflow model and a resource reservation mechanism, enabling the simultaneous execution of varied tasks. We demonstrate the implementation of AlabOS in a prototype autonomous materials laboratory, A-Lab, with around 3,500 samples synthesized over 1.5 years.
  arXiv  Detail & Related papers  (2024-05-22T18:59:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.