TMR-VLA:Vision-Language-Action Model for Magnetic Motion Control of Tri-leg Silicone-based Soft Robot

• URL: http://arxiv.org/abs/2603.00420v1
• Date: Sat, 28 Feb 2026 02:39:41 GMT
• Title: TMR-VLA:Vision-Language-Action Model for Magnetic Motion Control of Tri-leg Silicone-based Soft Robot
• Abstract summary: We developed a trileg magnetically driven soft robot (TMR) whose multi-legged design enables more flexible gaits and diverse motion patterns.<n>For the silicone made of reconfigurable soft robots, its navigation ability can be separated into sequential motions.<n>We introduced TMR-VLA, an end-to-end multi-modal system for a trileg magnetic soft robot capable of performing hybrid motion types.
• Score: 16.083189520931363
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In-vivo environments, magnetically actuated soft robots offer advantages such as wireless operation and precise control, showing promising potential for painless detection and therapeutic procedures. We developed a trileg magnetically driven soft robot (TMR) whose multi-legged design enables more flexible gaits and diverse motion patterns. For the silicone made of reconfigurable soft robots, its navigation ability can be separated into sequential motions, namely squatting, rotation, lifting a leg, walking and so on. Its motion and behavior depend on its bending shapes. To bridge motion type description and specific low-level voltage control, we introduced TMR-VLA, an end-to-end multi-modal system for a trileg magnetic soft robot capable of performing hybrid motion types, which is promising for developing a navigation ability by adapting its shape to language-constrained motion types. The TMR-VLA deploys embodied endoluminal localization ability from EndoVLA, and fuses sequential frames and natural language commands as input. Low-level voltage output is generated based on the current observation state and specific motion type description. The result shows the TMR-VLA can predict how the voltage applied to TMR will change the dynamics of a silicon-made soft robot. The TMR-VLA reached a 74% average success rate.

Related papers

• A Continual Learning Framework for Adaptive Control of Modular Soft Robots [4.374184567811783]
  Soft robots have attracted significant attention in applications such as medical intervention, rehabilitation, and robotic manipulation.<n>MSRs represent an emerging class of robotic systems with highly deformable and reconfigurable structures capable of performing complex tasks.<n>Existing approaches typically require controllers to be retrained from scratch whenever the robot morphology changes.<n>We propose a continual learning inspired control framework capable of incrementally adapting to changes in robot morphology.
  arXiv  Detail & Related papers  (2026-07-07T19:11:07Z)
• T-Rex: Tactile-Reactive Dexterous Manipulation [139.71263755530654]
  We introduce a variable-rate Mixture-of-Transformers (MoT) architecture equipped with a novel temporal tactile VQ-VAE encoder.<n>We demonstrate the effectiveness of tactile-reactive policies on 12 manipulation tasks requiring delicate force control and deformable object manipulation.
  arXiv  Detail & Related papers  (2026-06-15T17:59:55Z)
• MotionVLA: Vision-Language-Action Model for Humanoid Motion [54.785960777274276]
  We present DSFT, a dual-stream frequency tokenizer that separates motion into Base and physical streams.<n>We also present MotionVLA, a Qwen3.5-based model that arranges Base and physical tokens in a unified sequence.
  arXiv  Detail & Related papers  (2026-06-13T06:10:48Z)
• Mag-VLA: Vision-Language-Action Model for Bimanual Magnetically Actuated Microrobot Manipulation [6.712605140495474]
  We propose Mag-VLA, a vision-language-action (VLA) model for dexterous magnetic microrobot manipulation.<n>Our framework adapts a Qwen2.5-VL-7B backbone using Low-Rank Adaptation (LoRA) to process visual observations and language instructions for action prediction.<n>In real-robot experiments, Mag-VLA achieves a 90% approach success rate across all tasks and transport success rates of 80%, 70%, and 50% as task difficulty increases.
  arXiv  Detail & Related papers  (2026-05-27T13:44:00Z)
• RotVLA: Rotational Latent Action for Vision-Language-Action Model [54.22746299071677]
  We introduce RotVLA, a VLA framework built on a continuous rotational latent action representation.<n>Latent actions are modeled as elements of SO(n), providing continuity, compositionality, and structured geometry aligned with real-world action dynamics.<n>RotVLA consists of a VLM backbone and a flow-matching action head, pretrained on large-scale cross-embodiment robotic datasets and human videos.
  arXiv  Detail & Related papers  (2026-05-13T11:58:02Z)
• Learning Physics from Pretrained Video Models: A Multimodal Continuous and Sequential World Interaction Models for Robotic Manipulation [63.04810454548667]
  We introduce PhysGen, a scalable continuous and sequential world interaction framework to solve robotic manipulation tasks.<n>By treating the pretrained video model as a proxy for a physics simulator, PhysGen models the dynamic interplay between the external environment and robot actions.<n>We introduce a multimodal continuous representation that unifies video and action into shared physical tokens, bridging the gap between discrete video generation and continuous robotic control.
  arXiv  Detail & Related papers  (2026-02-18T14:58:18Z)
• EchoMotion: Unified Human Video and Motion Generation via Dual-Modality Diffusion Transformer [64.69014756863331]
  We introduce EchoMotion, a framework designed to model the joint distribution of appearance and human motion.<n>We also propose MVS-RoPE, which offers unified 3D positional encoding for both video and motion tokens.<n>Our findings reveal that explicitly representing human motion is to appearance, significantly boosting the coherence and plausibility of human-centric video generation.
  arXiv  Detail & Related papers  (2025-12-21T17:08:14Z)
• MiVLA: Towards Generalizable Vision-Language-Action Model with Human-Robot Mutual Imitation Pre-training [102.850162490626]
  We propose MiVLA, a vision-language-action model empowered by human-robot mutual imitation pre-training.<n>We show that MiVLA achieves strong improved generalization capability, outperforming state-of-the-art VLAs.
  arXiv  Detail & Related papers  (2025-12-17T12:59:41Z)
• MagBotSim: Physics-Based Simulation and Reinforcement Learning Environments for Magnetic Robotics [0.0]
  MagBotSim is a physics-based simulation for magnetic levitation systems.<n>This work lays the foundation for next generation manufacturing systems powered by Magnetic Robotics.
  arXiv  Detail & Related papers  (2025-11-20T08:53:54Z)
• FreeAction: Training-Free Techniques for Enhanced Fidelity of Trajectory-to-Video Generation [50.39748673817223]
  We introduce two training-free, inference-time techniques that fully exploit explicit action parameters in robot video generation.<n>First, action-scaled classifier-free guidance dynamically modulates guidance strength in proportion to action magnitude, enhancing controllability over motion intensity.<n>Second, action-scaled noise truncation adjusts the distribution of initially sampled noise to better align with the desired motion dynamics.
  arXiv  Detail & Related papers  (2025-09-29T03:30:40Z)
• VimoRAG: Video-based Retrieval-augmented 3D Motion Generation for Motion Language Models [110.32291962407078]
  VimoRAG is a video-based retrieval-augmented motion generation framework for motion large language models.<n>We develop an effective motion-centered video retrieval model and mitigate the issue of error propagation caused by suboptimal retrieval results.<n> Experimental results show that VimoRAG significantly boosts the performance of motion LLMs constrained to text-only input.
  arXiv  Detail & Related papers  (2025-08-16T15:31:14Z)
• Phoenix: A Motion-based Self-Reflection Framework for Fine-grained Robotic Action Correction [10.38090975412416]
  Building a generalizable self-correction system is crucial for robots to recover from failures.<n>We build the Phoenix framework, which leverages motion instruction as a bridge to connect high-level semantic reflection with low-level robotic action correction.<n>Experiments conducted in both the RoboMimic simulation and real-world scenarios prove the superior generalization and robustness of our framework.
  arXiv  Detail & Related papers  (2025-04-20T12:30:43Z)
• Moto: Latent Motion Token as the Bridging Language for Learning Robot Manipulation from Videos [101.26467307473638]
  We introduce Moto, which converts video content into latent Motion Token sequences by a Latent Motion Tokenizer.<n>We pre-train Moto-GPT through motion token autoregression, enabling it to capture diverse visual motion knowledge.<n>To transfer learned motion priors to real robot actions, we implement a co-fine-tuning strategy that seamlessly bridges latent motion token prediction and real robot control.
  arXiv  Detail & Related papers  (2024-12-05T18:57:04Z)
• Astrocyte Regulated Neuromorphic Central Pattern Generator Control of Legged Robotic Locomotion [7.828170373014957]
  This paper introduces an astrocyte regulated Spiking Neural Network (SNN)-based CPG for learning locomotion gait through Reward-Modulated STDP for quadruped robots.<n>The SNN-based CPG is simulated on a multi-object physics simulation platform resulting in the emergence of a trotting gait while running the robot on flat ground.<n>$23.3times$ computational power savings is observed in comparison to a state-of-the-art reinforcement learning based robot control algorithm.
  arXiv  Detail & Related papers  (2023-12-25T20:33:16Z)
• Automated Gait Generation For Walking, Soft Robotic Quadrupeds [6.005998680766498]
  Gait generation for soft robots is challenging due to the nonlinear dynamics and high dimensional input spaces of soft actuators. We present a sample-efficient, simulation free, method for self-generating soft robot gaits. This is the first demonstration of completely autonomous gait generation in a soft robot.
  arXiv  Detail & Related papers  (2023-09-30T21:31:30Z)
• MotionGPT: Finetuned LLMs Are General-Purpose Motion Generators [108.67006263044772]
  This paper presents a Motion General-Purpose generaTor (MotionGPT) that can use multimodal control signals. We first quantize multimodal control signals into discrete codes and then formulate them in a unified prompt instruction. Our MotionGPT demonstrates a unified human motion generation model with multimodal control signals by tuning a mere 0.4% of LLM parameters.
  arXiv  Detail & Related papers  (2023-06-19T12:58:17Z)
• Learning Low-Frequency Motion Control for Robust and Dynamic Robot Locomotion [10.838285018473725]
  We demonstrate robust and dynamic locomotion with a learned motion controller executing at as low as 8 Hz on a real ANYmal C quadruped. The robot is able to robustly and repeatably achieve a high heading velocity of 1.5 m/s, traverse uneven terrain, and resist unexpected external perturbations.
  arXiv  Detail & Related papers  (2022-09-29T15:55:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.