EchoAgent: Towards Reliable Echocardiography Interpretation with "Eyes","Hands" and "Minds"

• URL: http://arxiv.org/abs/2604.05541v1
• Date: Tue, 07 Apr 2026 07:39:11 GMT
• Title: EchoAgent: Towards Reliable Echocardiography Interpretation with "Eyes","Hands" and "Minds"
• Abstract summary: We propose EchoAgent, an agentic system tailored for end-to-end Echo interpretation.<n>It achieves a fully coordinated eyes-hands-minds workflow that learns, observes, operates, and reasons like a cardiac sonographer.<n>We evaluate EchoAgent on CAMUS and MIMIC-EchoQA datasets, which cover 48 distinct echocardiographic views.
• Score: 11.13999636495972
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reliable interpretation of echocardiography (Echo) is crucial for assessing cardiac function, which demands clinicians to synchronously orchestrate multiple capabilities, including visual observation (eyes), manual measurement (hands), and expert knowledge learning and reasoning (minds). While current task-specific deep-learning approaches and multimodal large language models have demonstrated promise in assisting Echo analysis through automated segmentation or reasoning, they remain focused on restricted skills, i.e., eyes-hands or eyes-minds, thereby limiting clinical reliability and utility. To address these issues, we propose EchoAgent, an agentic system tailored for end-to-end Echo interpretation, which achieves a fully coordinated eyes-hands-minds workflow that learns, observes, operates, and reasons like a cardiac sonographer. First, we introduce an expertise-driven cognition engine where our agent can automatically assimilate credible Echo guidelines into a structured knowledge base, thus constructing an Echo-customized mind. Second, we devise a hierarchical collaboration toolkit to endow EchoAgent with eyes-hands, which can automatically parse Echo video streams, identify cardiac views, perform anatomical segmentation, and quantitative measurement. Third, we integrate the perceived multimodal evidence with the exclusive knowledge base into an orchestrated reasoning hub to conduct explainable inferences. We evaluate EchoAgent on CAMUS and MIMIC-EchoQA datasets, which cover 48 distinct echocardiographic views spanning 14 cardiac anatomical regions. Experimental results show that EchoAgent achieves optimal performance across diverse structure analyses, yielding overall accuracy of up to 80.00%. Importantly, EchoAgent empowers a single system with abilities to learn, observe, operate and reason like an echocardiologist, which holds great promise for reliable Echo interpretation.

Related papers

• Echo-α: Large Agentic Multimodal Reasoning Model for Ultrasound Interpretation [76.6507710204181]
  We present Echo-, an agentic multimodal reasoning model for ultrasound interpretation.<n> Echo- is trained to coordinate organ-specific detector outputs, integrate them with global visual context, and convert the resulting evidence into grounded diagnostic decisions.<n>Results suggest that agentic multimodal reasoning can turn specialized detectors into verifiable clinical evidence.
  arXiv  Detail & Related papers  (2026-04-30T15:31:00Z)
• Evidence-Based Actor-Verifier Reasoning for Echocardiographic Agents [25.763431177629638]
  Visual language models (VLM) have opened a new avenue for building ultrasound understanding systems for clinical decision support.<n>We propose EchoTrust, an evidence-driven Actor-Verifier framework for trustworthy reasoning in echocardiography VLM-based agents.
  arXiv  Detail & Related papers  (2026-04-07T18:26:46Z)
• Echo2ECG: Enhancing ECG Representations with Cardiac Morphology from Multi-View Echos [53.368813255127115]
  Existing self-supervised methods suffer from a representational mismatch by aligning ECGs to single-view Echos.<n>We propose Echo2ECG, a multimodal self-supervised learning framework that enriches ECG representations with the heart's morphological structure captured in multi-view Echos.<n>We evaluate Echo2ECG as an ECG feature extractor on two clinically relevant tasks.
  arXiv  Detail & Related papers  (2026-03-09T15:39:57Z)
• Deep Expert Injection for Anchoring Retinal VLMs with Domain-Specific Knowledge [66.67024684187915]
  Large Vision Language Models (LVLMs) show immense potential for automated ophthalmic diagnosis.<n>Their clinical deployment is severely hindered by lacking domain-specific knowledge.<n>EyExIn is a framework designed to anchor retinal VLMs with expert knowledge via a Deep Expert Injection mechanism.
  arXiv  Detail & Related papers  (2026-03-07T09:43:49Z)
• Echo-CoPilot: A Multi-View, Multi-Task Agent for Echocardiography Interpretation and Reporting [8.162197738994479]
  We introduce Echo-CoPilot, a multi-view, multi-task agent that uses a large language model to orchestrate specialized echocardiography tools.<n>Within a ReAct-style loop, the agent decomposes clinician queries, invokes tools for view recognition, cardiac structure segmentation, measurement and disease prediction, and report synthesis.<n>We evaluate Echo-CoPilot on the public MIMIC-EchoQA benchmark, where it achieves an accuracy of 50.8%, outperforming both general-purpose and biomedical video vision-language models.
  arXiv  Detail & Related papers  (2025-12-06T23:27:54Z)
• EchoAgent: Guideline-Centric Reasoning Agent for Echocardiography Measurement and Interpretation [23.197431495208672]
  EchoAgent is a framework that enables structured, interpretable automation for echocardiographic video analysis.<n>It orchestrates specialized vision tools under Large Language Model (LLM) control to perform temporal localization, spatial measurement, and clinical interpretation.<n>It achieves accurate, interpretable results despite added complexity oftemporal video analysis.
  arXiv  Detail & Related papers  (2025-11-17T22:06:12Z)
• EchoWorld: Learning Motion-Aware World Models for Echocardiography Probe Guidance [79.66329903007869]
  We present EchoWorld, a motion-aware world modeling framework for probe guidance.<n>It encodes anatomical knowledge and motion-induced visual dynamics.<n>It is trained on more than one million ultrasound images from over 200 routine scans.
  arXiv  Detail & Related papers  (2025-04-17T16:19:05Z)
• EchoPrime: A Multi-Video View-Informed Vision-Language Model for Comprehensive Echocardiography Interpretation [1.0840985826142429]
  We introduce EchoPrime, a multi-view, view-informed, video-based vision-language foundation model trained on over 12 million video-report pairs. With retrieval-augmented interpretation, EchoPrime integrates information from all echocardiogram videos in a comprehensive study. In datasets from two independent healthcare systems, EchoPrime achieves state-of-the art performance on 23 diverse benchmarks of cardiac form and function.
  arXiv  Detail & Related papers  (2024-10-13T03:04:22Z)
• Show from Tell: Audio-Visual Modelling in Clinical Settings [58.88175583465277]
  We consider audio-visual modelling in a clinical setting, providing a solution to learn medical representations without human expert annotation. A simple yet effective multi-modal self-supervised learning framework is proposed for this purpose. The proposed approach is able to localise anatomical regions of interest during ultrasound imaging, with only speech audio as a reference.
  arXiv  Detail & Related papers  (2023-10-25T08:55:48Z)
• Intelligent Robotic Sonographer: Mutual Information-based Disentangled Reward Learning from Few Demonstrations [42.731081399649916]
  This work proposes an intelligent robotic sonographer to autonomously "explore" target anatomies and navigate a US probe to a relevant 2D plane by learning from the expert. The underlying high-level physiological knowledge from experts is inferred by a neural reward function. The proposed advanced framework can robustly work on a variety of seen and unseen phantoms as well as in-vivo human carotid data.
  arXiv  Detail & Related papers  (2023-07-07T16:30:50Z)
• Generalized Organ Segmentation by Imitating One-shot Reasoning using Anatomical Correlation [55.1248480381153]
  We propose OrganNet which learns a generalized organ concept from a set of annotated organ classes and then transfer this concept to unseen classes. We show that OrganNet can effectively resist the wide variations in organ morphology and produce state-of-the-art results in one-shot segmentation task.
  arXiv  Detail & Related papers  (2021-03-30T13:41:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.