Mag-VLA: Vision-Language-Action Model for Bimanual Magnetically Actuated Microrobot Manipulation

• URL: http://arxiv.org/abs/2605.28486v1
• Date: Wed, 27 May 2026 13:44:00 GMT
• Title: Mag-VLA: Vision-Language-Action Model for Bimanual Magnetically Actuated Microrobot Manipulation
• Abstract summary: We propose Mag-VLA, a vision-language-action (VLA) model for dexterous magnetic microrobot manipulation.<n>Our framework adapts a Qwen2.5-VL-7B backbone using Low-Rank Adaptation (LoRA) to process visual observations and language instructions for action prediction.<n>In real-robot experiments, Mag-VLA achieves a 90% approach success rate across all tasks and transport success rates of 80%, 70%, and 50% as task difficulty increases.
• Score: 6.712605140495474
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Magnetically actuated microrobots have been used as wireless, non-contact manipulation tools at microscales, making them promising for minimally invasive applications. However, their control remains challenging due to indirect actuation, limited sensing, and nonlinear magnetic interactions. In this work, we propose Mag-VLA, a vision-language-action (VLA) model for dexterous magnetic microrobot manipulation using two robotic arms with mounted magnets for dynamic magnetic-field construction. Bimanual coordination enables capabilities such as microrobot reorientation that are difficult or infeasible with a single arm, but it also introduces coupled control challenges, as the policy must generate coordinated trajectories for both actuators within a shared workspace. Our framework adapts a Qwen2.5-VL-7B backbone using Low-Rank Adaptation (LoRA) to process visual observations and language instructions for action prediction. To capture task progression, we introduce a motion-aware phase classifier and a phase-conditioned Action Chunking Transformer (ACT) decoder for temporally coherent multi-step control. We further construct a teleoperated magnetic microrobot manipulation dataset covering three task configurations. Ablation studies show that the ACT-based decoder substantially outperforms alternative generative action heads. In real-robot experiments, Mag-VLA achieves a 90% approach success rate across all tasks and transport success rates of 80%, 70%, and 50% as task difficulty increases. These results demonstrate that hierarchical VLA modeling provides a promising framework for magnetic microrobot manipulation.

Related papers

• RC-NF: Robot-Conditioned Normalizing Flow for Real-Time Anomaly Detection in Robotic Manipulation [68.7948300643741]
  We propose Robot-Conditioned Normalizing Flow (RC-NF), a real-time monitoring model for robotic anomaly detection and intervention.<n>RC-NF decouples the processing of task-aware robot and object states within the normalizing flow.<n>It achieves state-of-the-art performance across all anomaly types compared to previous methods in monitoring robotic tasks.
  arXiv  Detail & Related papers  (2026-03-11T10:14:37Z)
• TMR-VLA:Vision-Language-Action Model for Magnetic Motion Control of Tri-leg Silicone-based Soft Robot [16.083189520931363]
  We developed a trileg magnetically driven soft robot (TMR) whose multi-legged design enables more flexible gaits and diverse motion patterns.<n>For the silicone made of reconfigurable soft robots, its navigation ability can be separated into sequential motions.<n>We introduced TMR-VLA, an end-to-end multi-modal system for a trileg magnetic soft robot capable of performing hybrid motion types.
  arXiv  Detail & Related papers  (2026-02-28T02:39:41Z)
• MicroPush: A Simulator and Benchmark for Contact-Rich Cell Pushing and Assembly with a Magnetic Rolling Microrobot [0.6053202932413693]
  We present MicroPush, an open-source simulator and benchmark suite for magnetic rolling microrobots in cluttered 2D scenes.<n>MicroPush combines an overdamped interaction model with contact-aware stick--slip effects, lightweight near-field damping, optional Poiseuille background flow, and a calibrated mapping from actuation frequency to free-space rolling speed.
  arXiv  Detail & Related papers  (2026-02-27T02:16:26Z)
• ForceVLA: Enhancing VLA Models with a Force-aware MoE for Contact-rich Manipulation [62.58034332427291]
  ForceVLA is a novel end-to-end manipulation framework.<n>It treats external force sensing as a first-class modality within VLA systems.
  arXiv  Detail & Related papers  (2025-05-28T09:24:25Z)
• HAMSTER: Hierarchical Action Models For Open-World Robot Manipulation [54.03004125910057]
  We show that hierarchical vision-language-action models can be more effective in utilizing off-domain data than standard monolithic VLA models.<n>We show that, with the hierarchical design, the high-level VLM can transfer across significant domain gaps between the off-domain finetuning data and real-robot testing scenarios.
  arXiv  Detail & Related papers  (2025-02-08T07:50:22Z)
• FAST: Efficient Action Tokenization for Vision-Language-Action Models [98.15494168962563]
  We propose a new compression-based tokenization scheme for robot actions, based on the discrete cosine transform.<n>Based on FAST, we release FAST+, a universal robot action tokenizer, trained on 1M real robot action trajectories.
  arXiv  Detail & Related papers  (2025-01-16T18:57:04Z)
• RobotDiffuse: Motion Planning for Redundant Manipulator based on Diffusion Model [13.110235244912474]
  Redundant manipulators offer enhanced kinematic performance and versatility.<n>Motion planning for these manipulators is challenging due to increased DOFs and complex, dynamic environments.<n>This paper introduces RobotDiffuse, a diffusion model-based approach for motion planning in redundant manipulators.
  arXiv  Detail & Related papers  (2024-12-27T07:34:54Z)
• LLaRA: Supercharging Robot Learning Data for Vision-Language Policy [56.505551117094534]
  We introduce LLaRA: Large Language and Robotics Assistant, a framework that formulates robot action policy as visuo-textual conversations.<n>First, we present an automated pipeline to generate conversation-style instruction tuning data for robots from existing behavior cloning datasets.<n>We show that a VLM finetuned with a limited amount of such datasets can produce meaningful action decisions for robotic control.
  arXiv  Detail & Related papers  (2024-06-28T17:59:12Z)
• Smart Magnetic Microrobots Learn to Swim with Deep Reinforcement Learning [0.0]
  Deep reinforcement learning is a promising method of autonomously developing robust controllers for creating smart microrobots. Here, we report the development of a smart helical magnetic hydrogel microrobot that used the soft actor critic reinforcement learning algorithm to autonomously derive a control policy. The reinforcement learning agent learned successful control policies with fewer than 100,000 training steps, demonstrating sample efficiency for fast learning.
  arXiv  Detail & Related papers  (2022-01-14T18:42:18Z)
• Towards Safe Control of Continuum Manipulator Using Shielded Multiagent Reinforcement Learning [1.2647816797166165]
  The control of the robot is formulated as a one-DoF, one agent problem in the MADQN framework to improve the learning efficiency. Shielded MADQN enabled the robot to perform point and trajectory tracking with submillimeter root mean square errors under external loads.
  arXiv  Detail & Related papers  (2021-06-15T05:55:05Z)
• ReLMoGen: Leveraging Motion Generation in Reinforcement Learning for Mobile Manipulation [99.2543521972137]
  ReLMoGen is a framework that combines a learned policy to predict subgoals and a motion generator to plan and execute the motion needed to reach these subgoals. Our method is benchmarked on a diverse set of seven robotics tasks in photo-realistic simulation environments. ReLMoGen shows outstanding transferability between different motion generators at test time, indicating a great potential to transfer to real robots.
  arXiv  Detail & Related papers  (2020-08-18T08:05:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.