US-VLA: An Ultrasound Vision-Language-Action Model for Embodied Abdomina

• URL: http://arxiv.org/abs/2608.16074v1
• Date: Mon, 17 Aug 2026 04:12:21 GMT
• Title: US-VLA: An Ultrasound Vision-Language-Action Model for Embodied Abdomina
• Abstract summary: We propose an ultrasound vision-language-action model (US-VLA) for automated ultrasound scanning.<n>US-VLA encodes clinical semantic goals and generates sequential probe manipulation actions under real-time ultrasound feedback.<n>Experiments demonstrate that US-VLA achieves competitive performance in ultrasound probe manipulation tasks.
• Score: 18.156551505447226
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Artificial intelligence-assisted ultrasound scanning enhances diagnostic reliability and efficiency by providing real-time guidance for standardized image acquisition and reducing operator dependence. However, existing reinforcement learning and learning-assisted ultrasound scanning methods typically rely on carefully designed reward functions or extensive interaction data, which limits their generalization ability and stability across different devices, patient populations, and complex clinical scenarios. To address these challenges, we propose an ultrasound vision-language-action model (US-VLA) for automated ultrasound scanning that explicitly encodes clinical semantic goals and generates sequential probe manipulation actions under real-time ultrasound feedback. In particular, we first design an ultrasound-aware expert fusion module to jointly integrate ultrasound observations with auxiliary contextual information, enabling semantic ultrasound feedback to effectively guide the scanning process. Then, we construct US-VLA-Data, a real-world dataset covering liver and kidney examinations, which includes five clinically defined standard planes and comprises 320 expert scanning trajectories with approximately 80,000 synchronized timesteps. Extensive experiments demonstrate that US-VLA achieves competitive performance in ultrasound probe manipulation tasks, indicating its effectiveness and promising generalization within the evaluated abdominal ultrasound setting. The source code is available at https://github.com/VMVLab/US-VLA.

Related papers

• ForceU-VLA: A Force-Aware Vision-Language-Action Model for Embodied Ultrasound Scanning [17.732925254008943]
  Embodied intelligent ultrasound scanning enables the automation and standardization of the ultrasound examination process.<n>Existing methods suffer from loosely coupled modeling between force and ultrasound modalities.<n>We propose ForceU-VLA, a force-aware Vision-Language-Action model for autonomous embodied ultrasound scanning.
  arXiv  Detail & Related papers  (2026-08-15T03:39:40Z)
• TRUST: Efficient Abdominal Trauma Recognition via Image-to-Ultrasound-Video Transfer Learning [9.697220089288564]
  We propose a scan-aware PEIVTL framework that explicitly models fine-temporal variations to enable reliable ultrasound video understanding.<n>Experiments on in-house ultrasound trauma datasets demonstrate that TRUST outperforms state-of-the-art methods by 9.63% with superior computational efficiency.
  arXiv  Detail & Related papers  (2026-06-26T07:06:34Z)
• VA-Adapter: Adapting Ultrasound Foundation Model to Echocardiography Probe Guidance [57.43511837589102]
  We adapt medical knowledge learned by foundation models from vast datasets to the probe guidance task.<n>We meticulously design a parameter-efficient Vision-Action Adapter (VA-Adapter) to enable foundation model's image encoder to encode vision-action sequences.<n>With built-in sequential reasoning capabilities in a compact design, the VA-Adapter enables a pre-trained ultrasound foundation model to learn precise probe adjustment strategies.
  arXiv  Detail & Related papers  (2025-10-08T09:38:30Z)
• EchoVLM: Dynamic Mixture-of-Experts Vision-Language Model for Universal Ultrasound Intelligence [9.731550105507457]
  We propose EchoVLM, a vision-language model specifically designed for ultrasound medical imaging.<n>The model employs a Mixture of Experts (MoE) architecture trained on data spanning seven anatomical regions.<n>EchoVLM achieved significant improvements of 10.15 and 4.77 points in BLEU-1 scores and ROUGE-1 scores respectively.
  arXiv  Detail & Related papers  (2025-09-18T14:07:53Z)
• A Fully Open and Generalizable Foundation Model for Ultrasound Clinical Applications [77.3888788549565]
  We present EchoCare, a novel ultrasound foundation model for generalist clinical use.<n>We developed EchoCare via self-supervised learning on our curated, publicly available, large-scale dataset EchoCareData.<n>With minimal training, EchoCare outperforms state-of-the-art comparison models across 10 representative ultrasound benchmarks.
  arXiv  Detail & Related papers  (2025-09-15T10:05:31Z)
• Patient-Specific Real-Time Segmentation in Trackerless Brain Ultrasound [35.526097492693864]
  Intraoperative ultrasound (iUS) imaging has the potential to improve surgical outcomes in brain surgery. But its interpretation is challenging, even for expert neurosurgeons. In this work, we designed the first patient-specific framework that performs brain tumor segmentation in trackerless iUS.
  arXiv  Detail & Related papers  (2024-05-16T10:07:30Z)
• Enhancing Surgical Robots with Embodied Intelligence for Autonomous Ultrasound Scanning [24.014073238400137]
  Ultrasound robots are increasingly used in medical diagnostics and early disease screening. Current ultrasound robots lack the intelligence to understand human intentions and instructions. We propose a novel Ultrasound Embodied Intelligence system that equips ultrasound robots with the large language model and domain knowledge.
  arXiv  Detail & Related papers  (2024-05-01T11:39:38Z)
• CathFlow: Self-Supervised Segmentation of Catheters in Interventional Ultrasound Using Optical Flow and Transformers [66.15847237150909]
  We introduce a self-supervised deep learning architecture to segment catheters in longitudinal ultrasound images. The network architecture builds upon AiAReSeg, a segmentation transformer built with the Attention in Attention mechanism. We validated our model on a test dataset, consisting of unseen synthetic data and images collected from silicon aorta phantoms.
  arXiv  Detail & Related papers  (2024-03-21T15:13:36Z)
• AiAReSeg: Catheter Detection and Segmentation in Interventional Ultrasound using Transformers [75.20925220246689]
  endovascular surgeries are performed using the golden standard of Fluoroscopy, which uses ionising radiation to visualise catheters and vasculature. This work proposes a solution using an adaptation of a state-of-the-art machine learning transformer architecture to detect and segment catheters in axial interventional Ultrasound image sequences.
  arXiv  Detail & Related papers  (2023-09-25T19:34:12Z)
• Learning Robotic Ultrasound Scanning Skills via Human Demonstrations and Guided Explorations [12.894853456160924]
  We propose a learning-based approach to learn the robotic ultrasound scanning skills from human demonstrations. First, the robotic ultrasound scanning skill is encapsulated into a high-dimensional multi-modal model, which takes the ultrasound images, the pose/position of the probe and the contact force into account. Second, we leverage the power of imitation learning to train the multi-modal model with the training data collected from the demonstrations of experienced ultrasound physicians.
  arXiv  Detail & Related papers  (2021-11-02T14:38:09Z)
• Voice-assisted Image Labelling for Endoscopic Ultrasound Classification using Neural Networks [48.732863591145964]
  We propose a multi-modal convolutional neural network architecture that labels endoscopic ultrasound (EUS) images from raw verbal comments provided by a clinician during the procedure. Our results show a prediction accuracy of 76% at image level on a dataset with 5 different labels.
  arXiv  Detail & Related papers  (2021-10-12T21:22:24Z)
• Deep Learning for Ultrasound Beamforming [120.12255978513912]
  Beamforming, the process of mapping received ultrasound echoes to the spatial image domain, lies at the heart of the ultrasound image formation chain. Modern ultrasound imaging leans heavily on innovations in powerful digital receive channel processing. Deep learning methods can play a compelling role in the digital beamforming pipeline.
  arXiv  Detail & Related papers  (2021-09-23T15:15:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.