Bi-VLA: Bilateral Control-Based Imitation Learning via Vision-Language Fusion for Action Generation

• URL: http://arxiv.org/abs/2509.18865v1
• Date: Tue, 23 Sep 2025 10:02:16 GMT
• Title: Bi-VLA: Bilateral Control-Based Imitation Learning via Vision-Language Fusion for Action Generation
• Abstract summary: We propose Bilateral Control-Based Imitation Learning via Vision-Language Fusion for Action Generation (Bi-VLA)<n>Bi-VLA extends bilateral control-based imitation learning to handle more than one task within a single model.<n>We show Bi-VLA successfully interprets vision-language combinations and improves task success rates.
• Score: 0.3867363075280544
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose Bilateral Control-Based Imitation Learning via Vision-Language Fusion for Action Generation (Bi-VLA), a novel framework that extends bilateral control-based imitation learning to handle more than one task within a single model. Conventional bilateral control methods exploit joint angle, velocity, torque, and vision for precise manipulation but require task-specific models, limiting their generality. Bi-VLA overcomes this limitation by utilizing robot joint angle, velocity, and torque data from leader-follower bilateral control with visual features and natural language instructions through SigLIP and FiLM-based fusion. We validated Bi-VLA on two task types: one requiring supplementary language cues and another distinguishable solely by vision. Real-robot experiments showed that Bi-VLA successfully interprets vision-language combinations and improves task success rates compared to conventional bilateral control-based imitation learning. Our Bi-VLA addresses the single-task limitation of prior bilateral approaches and provides empirical evidence that combining vision and language significantly enhances versatility. Experimental results validate the effectiveness of Bi-VLA in real-world tasks. For additional material, please visit the website: https://mertcookimg.github.io/bi-vla/

Related papers

• QuoVLA: Quotient Space for Vision-Language-Action Models [51.02329790939691]
  Vision-Language-Action (VLA) models commonly adapt pretrained VLM latents to robot control by mapping visual observations and language instructions to continuous actions.<n>Our textitQuotient Theory for VLA shows that pretrained VLM latents are not action-insufficient but action-sufficient.<n>We propose QuoVLA, a quotient-space framework for VLA that compresses pretrained VLM latents into action-sufficient representations.
  arXiv  Detail & Related papers  (2026-05-24T06:28:53Z)
• PokeVLA: Empowering Pocket-Sized Vision-Language-Action Model with Comprehensive World Knowledge Guidance [24.102770290097435]
  PokeVLA is a lightweight model for embodied manipulation that infuses vision-language understanding into action learning.<n>Our framework introduces a two-stage training paradigm: first, we pre-train a compact vision-language model (PokeVLM) on a multimodal dataset of 2.4M samples.<n>Experiments demonstrate state-of-the-art performance on the LIBERO-Plus benchmark and in real-world deployment.
  arXiv  Detail & Related papers  (2026-04-22T17:58:19Z)
• DualCoT-VLA: Visual-Linguistic Chain of Thought via Parallel Reasoning for Vision-Language-Action Models [50.07453075750711]
  Vision-Language-Action (VLA) models map visual observations and language instructions directly to robotic actions.<n>Recent efforts have incorporated Chain-of-Thought (CoT) reasoning to endow VLA models with a thinking before acting'' capability.<n>We propose DualCoT-VLA, a visual-linguistic CoT method for VLA models with a parallel reasoning mechanism.
  arXiv  Detail & Related papers  (2026-03-23T17:59:25Z)
• DualVLA: Building a Generalizable Embodied Agent via Partial Decoupling of Reasoning and Action [62.70893433854428]
  We propose DualVLA, which enhances action performance through carefully designed post-training while still preserving reasoning capability.<n>Experiments show that DualVLA achieves an average success rate of 61.0 in SimplerEnv and an average score of 65.4 across eight competitive multimodal benchmarks.
  arXiv  Detail & Related papers  (2025-11-27T06:03:53Z)
• iFlyBot-VLA Technical Report [25.330744626382977]
  We introduce iFlyBot-VLA, a large-scale Vision-Language-Action (VLA) model trained under a novel framework.<n>The main contributions are listed as follows: (1) a latent action model thoroughly trained on large-scale human and robotic manipulation videos; (2) a dual-level action representation framework that jointly supervises both the Vision-Language Model (VLM) and the action expert during training; and (3) a mixed training strategy that combines robot trajectory data with general QA and spatial QA datasets.
  arXiv  Detail & Related papers  (2025-11-01T06:24:56Z)
• dVLA: Diffusion Vision-Language-Action Model with Multimodal Chain-of-Thought [66.78110237549087]
  Vision-Language-Action (VLA) models are emerging as a next-generation paradigm for robotics.<n>We introduce dVLA, a diffusion-based VLA that unifies visual perception, language reasoning, and robotic control in a single system.
  arXiv  Detail & Related papers  (2025-09-30T02:36:11Z)
• DAM: Dual Active Learning with Multimodal Foundation Model for Source-Free Domain Adaptation [53.323488295994395]
  Source-free active domain adaptation (SFADA) enhances knowledge transfer from a source model to an unlabeled target domain using limited manual labels selected via active learning.<n>We propose Dual Active learning with Multimodal (DAM) foundation model, a novel framework that integrates multimodal supervision from a ViL model to complement sparse human annotations.<n>Extensive experiments demonstrate that DAM consistently outperforms existing methods and sets a new state-of-the-art across multiple SFADA benchmarks and active learning strategies.
  arXiv  Detail & Related papers  (2025-09-29T15:06:56Z)
• CogVLA: Cognition-Aligned Vision-Language-Action Model via Instruction-Driven Routing & Sparsification [48.81250395291505]
  Recent Vision-Language-Action models require extensive post-training, resulting in high computational overhead.<n>We propose CogVLA, a framework that leverages instruction-driven routing and sparsification to improve both efficiency and performance.<n>CogVLA achieves state-of-the-art performance with success rates of 97.4% and 70.0%, respectively, while reducing training costs by 2.5-fold and decreasing inference latency by 2.8-fold compared to OpenVLA.
  arXiv  Detail & Related papers  (2025-08-28T17:50:58Z)
• Bi-LAT: Bilateral Control-Based Imitation Learning via Natural Language and Action Chunking with Transformers [16.724380665811044]
  Bi-LAT is a novel imitation learning framework that unifies bilateral control with natural language processing.<n>Bi-LAT learns to distinguish nuanced force requirements in real-world tasks.
  arXiv  Detail & Related papers  (2025-04-02T02:21:30Z)
• ChatVLA: Unified Multimodal Understanding and Robot Control with Vision-Language-Action Model [21.844214660424175]
  ChatVLA is a novel framework featuring Phased Alignment Training, which incrementally integrates multimodal data after initial control mastery, and a Mixture-of-Experts architecture to minimize task interference.<n>ChatVLA demonstrates competitive performance on visual question-answering datasets and significantly surpasses state-of-the-art vision-language-action (VLA) methods on multimodal understanding benchmarks.<n>Our findings highlight the potential of our unified framework for achieving both robust multimodal understanding and effective robot control.
  arXiv  Detail & Related papers  (2025-02-20T10:16:18Z)
• OpenVLA: An Open-Source Vision-Language-Action Model [131.74098076670103]
  We introduce OpenVLA, an open-source VLA trained on a diverse collection of 970k real-world robot demonstrations. OpenVLA shows strong results for generalist manipulation, outperforming closed models such as RT-2-X (55B) by 16.5% in absolute task success rate. We release model checkpoints, fine-tuning notebooks, and our PyTorch with built-in support for training VLAs at scale on Open X-Embodiment datasets.
  arXiv  Detail & Related papers  (2024-06-13T15:46:55Z)
• LION : Empowering Multimodal Large Language Model with Dual-Level Visual Knowledge [58.82222646803248]
  Multimodal Large Language Models (MLLMs) have endowed LLMs with the ability to perceive and understand multi-modal signals. Most of the existing MLLMs mainly adopt vision encoders pretrained on coarsely aligned image-text pairs, leading to insufficient extraction and reasoning of visual knowledge. We propose a dual-Level vIsual knedgeOwl eNhanced Multimodal Large Language Model (LION), which empowers the MLLM by injecting visual knowledge in two levels.
  arXiv  Detail & Related papers  (2023-11-20T15:56:44Z)
• Enabling Multimodal Generation on CLIP via Vision-Language Knowledge Distillation [79.72299298976525]
  We propose to augment a vision-language pre-training model with a textual pre-trained language model (PLM) via vision-language knowledge distillation (VLKD) Experiments show that the resulting model has strong zero-shot performance on multimodal generation tasks, such as open-ended visual question answering and image captioning. The original textual language understanding and generation ability of the PLM is maintained after VLKD, which makes our model versatile for both multimodal and unimodal tasks.
  arXiv  Detail & Related papers  (2022-03-12T09:33:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.