MARCUS: An agentic, multimodal vision-language model for cardiac diagnosis and management

• URL: http://arxiv.org/abs/2603.22179v1
• Date: Mon, 23 Mar 2026 16:42:11 GMT
• Title: MARCUS: An agentic, multimodal vision-language model for cardiac diagnosis and management
• Abstract summary: Current AI vision-language models are limited to single-modality inputs and are non-interactive.<n>We present MARCUS, an agentic vision-language system for end-to-end interpretation of electrocardiograms, echocardiograms, and cardiac magnetic resonance imaging (CMR) independently and as multimodal input.<n>MARCUS employs a hierarchical agentic architecture comprising modality-specific vision-language expert models, each integrating domain-trained visual encoders with multi-stage language model optimization, coordinated by a multimodal orchestrator.
• Score: 7.8540088642712815
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Cardiovascular disease remains the leading cause of global mortality, with progress hindered by human interpretation of complex cardiac tests. Current AI vision-language models are limited to single-modality inputs and are non-interactive. We present MARCUS (Multimodal Autonomous Reasoning and Chat for Ultrasound and Signals), an agentic vision-language system for end-to-end interpretation of electrocardiograms (ECGs), echocardiograms, and cardiac magnetic resonance imaging (CMR) independently and as multimodal input. MARCUS employs a hierarchical agentic architecture comprising modality-specific vision-language expert models, each integrating domain-trained visual encoders with multi-stage language model optimization, coordinated by a multimodal orchestrator. Trained on 13.5 million images (0.25M ECGs, 1.3M echocardiogram images, 12M CMR images) and our novel expert-curated dataset spanning 1.6 million questions, MARCUS achieves state-of-the-art performance surpassing frontier models (GPT-5 Thinking, Gemini 2.5 Pro Deep Think). Across internal (Stanford) and external (UCSF) test cohorts, MARCUS achieves accuracies of 87-91% for ECG, 67-86% for echocardiography, and 85-88% for CMR, outperforming frontier models by 34-45% (P<0.001). On multimodal cases, MARCUS achieved 70% accuracy, nearly triple that of frontier models (22-28%), with 1.7-3.0x higher free-text quality scores. Our agentic architecture also confers resistance to mirage reasoning, whereby vision-language models derive reasoning from unintended textual signals or hallucinated visual content. MARCUS demonstrates that domain-specific visual encoders with an agentic orchestrator enable multimodal cardiac interpretation. We release our models, code, and benchmark open-source.

Related papers

• ClinFusion: A Vision-Centric Multimodal LLM System for Holistic Medical Understanding [67.57715989349471]
  Multimodal large language models (MLLMs) hold immense potential to revolutionize clinical practice, yet deploying them in the medical domain is fundamentally a vision-centric challenge.<n>We introduce ClinFusion, a vision-centric MLLM designed for holistic medical understanding.<n>We show that ClinFusion sets a new state-of-the-art across a comprehensive suite of 2D and 3D multimodal medical benchmarks.
  arXiv  Detail & Related papers  (2026-07-27T17:59:49Z)
• Development of an automated, reliable, and clinically meaningful artificial intelligence (AI) tool for diagnosing cardiac disease from conventional cardiovascular magnetic resonance (CMR) images [30.83069805418805]
  We developed an automated data curation pipeline for CMR-based cardiovascular disease diagnosis.<n>Three vision foundation models (DINO, VSTedPT) were fine-tuned across these modalities in a two-stage approach.<n>Fine-tuned AI-models achieved high discriminative diagnostic performance on an independent test set.
  arXiv  Detail & Related papers  (2026-07-22T12:36:56Z)
• Disease-Centric Vision-Language Pretraining with Hybrid Visual Encoding for 3D Computed Tomography [75.7789001619107]
  Vision-language pre-training holds great promise for general-purpose medical AI.<n>We propose a tailored framework featuring three key components.<n>A diagnosis-aware prompt strategy employs real clinical phrases and aggregated disease prototypes to bridge the pre-training-inference gap.
  arXiv  Detail & Related papers  (2026-06-24T08:24:45Z)
• OphMAE: Bridging Volumetric and Planar Imaging with a Foundation Model for Adaptive Ophthalmological Diagnosis [13.730900699529698]
  Ophthalmic multimodal Masked Autoencoder (OphMAE) is a multi-imaging foundation model engineered to synergize the volumetric depth of 3D Optical Coherence Tomography ( OCT) with the planar context of 2D en face OCT.<n>The model demonstrated state-of-the-art performance, achieving an Area Under the Curve (AUC) of 96.9% for Age-related Macular Degeneration (AMD) and 97.2% for Diabetic Macular Edema (DME)<n>This work establishes a scalable and adaptable framework for ophthalmic AI, ensuring robust performance across different tasks.
  arXiv  Detail & Related papers  (2026-05-04T15:18:06Z)
• LUMINA: A Multi-Vendor Mammography Benchmark with Energy Harmonization Protocol [9.075009508165273]
  This dataset contains 1824 images from 468 patients with pathology-confirmed labels, BI-RADS assessments, and breast-density annotations.<n>LUMINA spans six acquisition systems and includes both high- and low-energy imaging styles.<n>We benchmark CNN and transformer models on three clinically relevant tasks: diagnosis, BI-RADS classification, and density estimation.
  arXiv  Detail & Related papers  (2026-03-15T22:41:40Z)
• KorMedMCQA-V: A Multimodal Benchmark for Evaluating Vision-Language Models on the Korean Medical Licensing Examination [16.50828571559655]
  KorMedMCQA-V is a Korean medical licensing-exam-style multimodal multiple-choice question answering benchmark.<n>The dataset consists of 1,534 questions with 2,043 associated images from Korean Medical Licensing Examinations.
  arXiv  Detail & Related papers  (2026-02-14T07:42:04Z)
• Enabling Ultra-Fast Cardiovascular Imaging Across Heterogeneous Clinical Environments with a Generalist Foundation Model and Multimodal Database [64.65360708629485]
  MMCMR-427K is the largest and most comprehensive multimodal cardiovascular magnetic resonance k-space database.<n> CardioMM is a reconstruction foundation model capable of adapting to heterogeneous fast CMR imaging scenarios.<n> CardioMM unifies semantic contextual understanding with physics-informed data consistency to deliver robust reconstructions.
  arXiv  Detail & Related papers  (2025-12-25T12:47:50Z)
• Hulu-Med: A Transparent Generalist Model towards Holistic Medical Vision-Language Understanding [112.46150793476603]
  We introduce Hulu-Med, a transparent, generalist medical Vision-Language Model (VLM)<n>Hulu-Med is trained on a curated corpus of 16.7 million samples, spanning 12 major anatomical systems and 14 medical imaging modalities.<n>Hulu-Med surpasses existing open-source models on 27 of 30 benchmarks and outperforms proprietary systems such as GPT-4o on 16 benchmarks.
  arXiv  Detail & Related papers  (2025-10-09T17:06:42Z)
• DKG-LLM : A Framework for Medical Diagnosis and Personalized Treatment Recommendations via Dynamic Knowledge Graph and Large Language Model Integration [0.0]
  We present the DKG-LLM framework for medical diagnosis and personalized treatment recommendations.<n>The framework integrates a dynamic knowledge graph (DKG) with the Grok 3 large language model.<n>The evaluation results show that DKG-LLM achieves a diagnostic accuracy of 84.19% and a treatment recommendation accuracy of 89.63%.
  arXiv  Detail & Related papers  (2025-08-08T10:04:40Z)
• Heartcare Suite: Multi-dimensional Understanding of ECG with Raw Multi-lead Signal Modeling [50.58126509704037]
  Heartcare Suite is a framework for fine-grained electrocardiogram (ECG) understanding.<n>Heartcare-220K is a high-quality, structured, and comprehensive multimodal ECG dataset.<n>Heartcare-Bench is a benchmark to guide the optimization of Medical Multimodal Large Language Models (Med-MLLMs) in ECG scenarios.
  arXiv  Detail & Related papers  (2025-06-06T07:56:41Z)
• Multi-Model Ensemble Approach for Accurate Bi-Atrial Segmentation in LGE-MRI of Atrial Fibrillation Patients [3.676588766498097]
  Atrial fibrillation (AF) is the most prevalent form of cardiac arrhythmia and is associated with increased morbidity and mortality. This work presents an ensemble approach that integrates multiple machine learning models, including Unet, ResNet, EfficientNet and VGG, to perform automatic bi-atrial segmentation from LGE-MRI data.
  arXiv  Detail & Related papers  (2024-09-24T13:33:46Z)
• RadGenome-Chest CT: A Grounded Vision-Language Dataset for Chest CT Analysis [56.57177181778517]
  RadGenome-Chest CT is a large-scale, region-guided 3D chest CT interpretation dataset based on CT-RATE. We leverage the latest powerful universal segmentation and large language models to extend the original datasets.
  arXiv  Detail & Related papers  (2024-04-25T17:11:37Z)
• Advancing COVID-19 Diagnosis with Privacy-Preserving Collaboration in Artificial Intelligence [79.038671794961]
  We launch the Unified CT-COVID AI Diagnostic Initiative (UCADI), where the AI model can be distributedly trained and independently executed at each host institution. Our study is based on 9,573 chest computed tomography scans (CTs) from 3,336 patients collected from 23 hospitals located in China and the UK.
  arXiv  Detail & Related papers  (2021-11-18T00:43:41Z)
• ECG Heartbeat Classification Using Multimodal Fusion [13.524306011331303]
  We propose two computationally efficient multimodal fusion frameworks for ECG heart beat classification. In MFF, we extracted features from penultimate layer of CNNs and fused them to get unique and interdependent information. We achieved classification accuracy of 99.7% and 99.2% on arrhythmia and MI classification, respectively.
  arXiv  Detail & Related papers  (2021-07-21T03:48:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.