Gastric-X: A Multimodal Multi-Phase Benchmark Dataset for Advancing Vision-Language Models in Gastric Cancer Analysis

• URL: http://arxiv.org/abs/2603.19516v2
• Date: Thu, 26 Mar 2026 11:12:17 GMT
• Title: Gastric-X: A Multimodal Multi-Phase Benchmark Dataset for Advancing Vision-Language Models in Gastric Cancer Analysis
• Abstract summary: Gastric-X is a large-scale benchmark for gastric cancer analysis providing 1.7K cases.<n>We systematically examine the capability of recent vision-language models on five core tasks.<n>We envision Gastric-X as a step toward aligning machine intelligence with the cognitive and evidential reasoning processes of physicians.
• Score: 10.801293378989
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Recent vision-language models (VLMs) have shown strong generalization and multimodal reasoning abilities in natural domains. However, their application to medical diagnosis remains limited by the lack of comprehensive and structured datasets that capture real clinical workflows. To advance the development of VLMs for clinical applications, particularly in gastric cancer, we introduce Gastric-X, a large-scale multimodal benchmark for gastric cancer analysis providing 1.7K cases. Each case in Gastric-X includes paired resting and dynamic CT scans, endoscopic image, a set of structured biochemical indicators, expert-authored diagnostic notes, and bounding box annotations of tumor regions, reflecting realistic clinical conditions. We systematically examine the capability of recent VLMs on five core tasks: Visual Question Answering (VQA), report generation, cross-modal retrieval, disease classification, and lesion localization. These tasks simulate critical stages of clinical workflow, from visual understanding and reasoning to multimodal decision support. Through this evaluation, we aim not only to assess model performance but also to probe the nature of VLM understanding: Can current VLMs meaningfully correlate biochemical signals with spatial tumor features and textual reports? We envision Gastric-X as a step toward aligning machine intelligence with the cognitive and evidential reasoning processes of physicians, and as a resource to inspire the development of next-generation medical VLMs.

Related papers

• Concept-Grounded Reasoning with Prompt-Driven Localization for Interpretable Structured Report Generation [66.62662330558906]
  CORAL is a multimodal framework that integrates spatial grounding and concept-level supervision into a unified reasoning process.<n>CORAL employs a prompt-driven medical segmentation model to localize lesions and predicts multi-class clinical attributes through a Concept Bottleneck module.<n>Experiments on BUS-CoT and IU X-ray datasets demonstrate consistent improvements in diagnostic accuracy, concept consistency, and report quality over strong general-purpose and medical MLLMs.
  arXiv  Detail & Related papers  (2026-09-14T10:21:23Z)
• NeoRed: A Knowledge-Logic-Alignment Multimodal Large Language Model for Neonatal Respiratory Disease Diagnosis [51.033331232872094]
  Neonatal respiratory diseases are a major cause of neonatal morbidity and mortality.<n>Existing Multimodal Large Language Models (MLLMs) face two key limitations in neonatal diagnosis.<n>We propose NeoRed, to the best of our knowledge, the first MLLM tailored for neonatal respiratory disease.
  arXiv  Detail & Related papers  (2026-09-03T08:24:38Z)
• MMBU: A Massive Multi-modal Biomedical Understanding Benchmark to Probe the Perception Capabilities of Vision-Language Models [83.50100003741628]
  Vision and language models (VLMs) hold immense promise to transform biomedical imaging.<n>We introduce the Massive Multimodal Biomedical Understanding (MMBU) benchmark.<n>It is the largest vision and language benchmark to date, covering 35 submodalities with rich structured metadata.
  arXiv  Detail & Related papers  (2026-06-04T20:24:47Z)
• Clinical Cognition Alignment for Gastrointestinal Diagnosis with Multimodal LLMs [63.535652574541764]
  Multimodal Large Language Models (MLLMs) have demonstrated remarkable potential in medical image analysis.<n>Their application in gastrointestinal endoscopy is currently hindered by two critical limitations.<n>We propose a novel Clinical-Cognitive-Aligned (CogAlign) framework to address these challenges.
  arXiv  Detail & Related papers  (2026-03-21T07:47:37Z)
• Sim4Seg: Boosting Multimodal Multi-disease Medical Diagnosis Segmentation with Region-Aware Vision-Language Similarity Masks [54.00822479127598]
  We introduce a medical vision-language task named Medical Diagnosis (MDS)<n>MDS aims to understand clinical queries for medical images and generate the corresponding segmentation masks as well as diagnostic results.<n>We propose Sim4Seg, a novel framework that improves the performance of diagnosis segmentation.
  arXiv  Detail & Related papers  (2025-11-10T03:22:42Z)
• A Narrative Review on Large AI Models in Lung Cancer Screening, Diagnosis, and Treatment Planning [8.431488361911754]
  Lung cancer remains one of the most prevalent and fatal diseases worldwide.<n>Recent advancements in large AI models have significantly enhanced medical image understanding and clinical decision-making.<n>This review systematically surveys the state-of-the-art in applying large AI models to lung cancer screening, diagnosis, prognosis, and treatment.
  arXiv  Detail & Related papers  (2025-06-08T17:42:24Z)
• EndoBench: A Comprehensive Evaluation of Multi-Modal Large Language Models for Endoscopy Analysis [62.00431604976949]
  EndoBench is the first comprehensive benchmark specifically designed to assess MLLMs across the full spectrum of endoscopic practice.<n>We benchmark 23 state-of-the-art models, including general-purpose, medical-specialized, and proprietary MLLMs.<n>Our experiments reveal: proprietary MLLMs outperform open-source and medical-specialized models overall, but still trail human experts.
  arXiv  Detail & Related papers  (2025-05-29T16:14:34Z)
• ChatEXAONEPath: An Expert-level Multimodal Large Language Model for Histopathology Using Whole Slide Images [19.661619004001654]
  We introduce an expert-level MLLM for histopathology using WSIs, dubbed as ChatEXAONEPath.<n>We demonstrate the ability of diagnosing the given histopathology images using ChatEXAONEPath with the acceptance rate of 62.9%.<n>Our proposed model can understand pan-cancer WSIs and clinical context from various cancer types.
  arXiv  Detail & Related papers  (2025-04-17T15:33:17Z)
• GMAI-MMBench: A Comprehensive Multimodal Evaluation Benchmark Towards General Medical AI [67.09501109871351]
  Large Vision-Language Models (LVLMs) are capable of handling diverse data types such as imaging, text, and physiological signals. GMAI-MMBench is the most comprehensive general medical AI benchmark with well-categorized data structure and multi-perceptual granularity to date. It is constructed from 284 datasets across 38 medical image modalities, 18 clinical-related tasks, 18 departments, and 4 perceptual granularities in a Visual Question Answering (VQA) format.
  arXiv  Detail & Related papers  (2024-08-06T17:59:21Z)
• Potential of Multimodal Large Language Models for Data Mining of Medical Images and Free-text Reports [51.45762396192655]
  Multimodal large language models (MLLMs) have recently transformed many domains, significantly affecting the medical field. Notably, Gemini-Vision-series (Gemini) and GPT-4-series (GPT-4) models have epitomized a paradigm shift in Artificial General Intelligence for computer vision. This study evaluated the performance of the Gemini, GPT-4, and 4 popular large models for an exhaustive evaluation across 14 medical imaging datasets.
  arXiv  Detail & Related papers  (2024-07-08T09:08:42Z)
• Foundational Models for Pathology and Endoscopy Images: Application for Gastric Inflammation [0.0]
  Foundation models (FM) are machine or deep learning models trained on diverse data and applicable to broad use cases. FM offer a promising solution to enhance the accuracy of endoscopy and its subsequent pathology image analysis. This review aims to provide a roadmap for researchers and practitioners in navigating the complexities of incorporating FM into clinical practice.
  arXiv  Detail & Related papers  (2024-06-26T10:51:44Z)
• Dr-LLaVA: Visual Instruction Tuning with Symbolic Clinical Grounding [53.629132242389716]
  Vision-Language Models (VLM) can support clinicians by analyzing medical images and engaging in natural language interactions. VLMs often exhibit "hallucinogenic" behavior, generating textual outputs not grounded in contextual multimodal information. We propose a new alignment algorithm that uses symbolic representations of clinical reasoning to ground VLMs in medical knowledge.
  arXiv  Detail & Related papers  (2024-05-29T23:19:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.