MedVeriSeg: Teaching MLLM-Based Medical Segmentation Models to Verify Query Validity Without Extra Training

• URL: http://arxiv.org/abs/2604.10242v2
• Date: Tue, 14 Apr 2026 07:00:34 GMT
• Title: MedVeriSeg: Teaching MLLM-Based Medical Segmentation Models to Verify Query Validity Without Extra Training
• Abstract summary: We propose MedVeriSeg, a training-free verification framework for LISA-like medical segmentation models.<n>Our key observation is that the similarity map between the [SEG] token feature and MLLM image features exhibits markedly different distribution patterns for true and false queries.<n>Experiments on a small-scale benchmark constructed from SA-Med2D-20M show that MedVeriSeg effectively rejects false-query segmentation requests.
• Score: 20.87443071413578
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite recent advances in MLLM-based medical image segmentation, existing LISA-like methods cannot reliably reject false queries and often produce hallucinated segmentation masks for absent targets. This limitation reduces practical reliability in both medical education and clinical use. In this work, we propose MedVeriSeg, a training-free verification framework that equips LISA-like medical segmentation models with the ability to identify and reject false queries which contain non-existent targets. Our key observation is that the similarity map between the [SEG] token feature and MLLM image features exhibits markedly different distribution patterns for true and false queries. Based on this, we introduce a Similarity Response Quality Scoring Module that characterizes the similarity map from three aspects: strength, compactness, and purity, producing an initial target-existence prediction. We further incorporate qualitative visual evidence by using GPT-4o to jointly assess the similarity heatmap and the results of Similarity Response Quality Scoring Module for final verification. Experiments on a small-scale benchmark constructed from SA-Med2D-20M show that MedVeriSeg effectively rejects false-query segmentation requests while maintaining reliable recognition of true queries.

Related papers

• From Reasoning to Pixels: Grounded Medical Multimodal LLMs for VQA and Segmentation [24.67847881046532]
  textscMedREAL (textbfMedical textbfREasoning-driven textbfAnswering and textbfLocalization) is a unified framework that seamlessly aligns linguistic reasoning with spatial grounding.<n>textscMedREAL significantly outperforms state-of-the-arts, achieving 68.49% gIoU and 70.47% cIoU on benchmark evaluations.
  arXiv  Detail & Related papers  (2026-08-27T09:23:16Z)
• CardioLens: Revealing the Clinical Reality Gap of MLLMs via Multi-Sequence Cardiac MRI Evaluations [16.42889396398046]
  We introduce CardioLens, a leakage-resistant evaluation testbed for multi-sequence Cardiovascular Magnetic Resonance (CMR)<n> CardioLens contains 473,896 slices and 13,494 verified QA pairs across 4D Cine, LGE, perfusion, and T2-weighted imaging.<n>Across 24 state-of-the-art MLLMs, CardioLens reveals a substantial clinical reality gap: models perform poorly overall, with performance degrading along the real CMR workflow.
  arXiv  Detail & Related papers  (2026-05-28T11:03:06Z)
• Lost in the Hype: Revealing and Dissecting the Performance Degradation of Medical Multimodal Large Language Models in Image Classification [14.247959730104085]
  multimodal large language models (MLLMs) have sparked an unprecedented wave of applications in the field of medical imaging analysis.<n>However, medical image classification reveals a sobering reality: state-of-the-art medical MLLMs consistently underperform compared to traditional deep learning models.<n>In this paper, we conduct extensive experiments on 14 open-source medical MLLMs across three representative image classification datasets.
  arXiv  Detail & Related papers  (2026-04-09T15:07:26Z)
• M3CoTBench: Benchmark Chain-of-Thought of MLLMs in Medical Image Understanding [66.78251988482222]
  Chain-of-Thought (CoT) reasoning has proven effective in enhancing large language models by encouraging step-by-step intermediate reasoning.<n>Current benchmarks for medical image understanding generally focus on the final answer while ignoring the reasoning path.<n>M3CoTBench aims to foster the development of transparent, trustworthy, and diagnostically accurate AI systems for healthcare.
  arXiv  Detail & Related papers  (2026-01-13T17:42:27Z)
• Proof of Concept for Mammography Classification with Enhanced Compactness and Separability Modules [0.0]
  This study presents a validation and extension of a recent methodological framework for medical image classification.<n>Using a Kaggle dataset that consolidates INbreast, MIAS, and InceptionM mammography collections, we compare a baseline CNN, ConvNeXt Tiny, and InceptionV3 backbones enriched with GAGM and SE modules.<n>Results confirm the effectiveness of GAGM and SE in enhancing feature discriminability and reducing false negatives.<n>In our experiments, however, the Feature Smoothing Loss did not yield measurable improvements under mammography classification conditions.
  arXiv  Detail & Related papers  (2025-12-06T21:36:05Z)
• Sim4Seg: Boosting Multimodal Multi-disease Medical Diagnosis Segmentation with Region-Aware Vision-Language Similarity Masks [54.00822479127598]
  We introduce a medical vision-language task named Medical Diagnosis (MDS)<n>MDS aims to understand clinical queries for medical images and generate the corresponding segmentation masks as well as diagnostic results.<n>We propose Sim4Seg, a novel framework that improves the performance of diagnosis segmentation.
  arXiv  Detail & Related papers  (2025-11-10T03:22:42Z)
• SemSim: Revisiting Weak-to-Strong Consistency from a Semantic Similarity Perspective for Semi-supervised Medical Image Segmentation [18.223854197580145]
  Semi-supervised learning (SSL) for medical image segmentation is a challenging yet highly practical task. We propose a novel framework based on FixMatch, named SemSim, powered by two appealing designs from semantic similarity perspective. We show that SemSim yields consistent improvements over the state-of-the-art methods across three public segmentation benchmarks.
  arXiv  Detail & Related papers  (2024-10-17T12:31:37Z)
• Contrastive Learning with Counterfactual Explanations for Radiology Report Generation [83.30609465252441]
  We propose a textbfCountertextbfFactual textbfExplanations-based framework (CoFE) for radiology report generation. Counterfactual explanations serve as a potent tool for understanding how decisions made by algorithms can be changed by asking what if'' scenarios. Experiments on two benchmarks demonstrate that leveraging the counterfactual explanations enables CoFE to generate semantically coherent and factually complete reports.
  arXiv  Detail & Related papers  (2024-07-19T17:24:25Z)
• Uncertainty-Aware Adapter: Adapting Segment Anything Model (SAM) for Ambiguous Medical Image Segmentation [20.557472889654758]
  The Segment Anything Model (SAM) gained significant success in natural image segmentation. Unlike natural images, many tissues and lesions in medical images have blurry boundaries and may be ambiguous. We propose a novel module called the Uncertainty-aware Adapter, which efficiently fine-tune SAM for uncertainty-aware medical image segmentation.
  arXiv  Detail & Related papers  (2024-03-16T14:11:54Z)
• Rethinking Semi-Supervised Medical Image Segmentation: A Variance-Reduction Perspective [51.70661197256033]
  We propose ARCO, a semi-supervised contrastive learning framework with stratified group theory for medical image segmentation. We first propose building ARCO through the concept of variance-reduced estimation and show that certain variance-reduction techniques are particularly beneficial in pixel/voxel-level segmentation tasks. We experimentally validate our approaches on eight benchmarks, i.e., five 2D/3D medical and three semantic segmentation datasets, with different label settings.
  arXiv  Detail & Related papers  (2023-02-03T13:50:25Z)
• Label Cleaning Multiple Instance Learning: Refining Coarse Annotations on Single Whole-Slide Images [83.7047542725469]
  Annotating cancerous regions in whole-slide images (WSIs) of pathology samples plays a critical role in clinical diagnosis, biomedical research, and machine learning algorithms development. We present a method, named Label Cleaning Multiple Instance Learning (LC-MIL), to refine coarse annotations on a single WSI without the need of external training data. Our experiments on a heterogeneous WSI set with breast cancer lymph node metastasis, liver cancer, and colorectal cancer samples show that LC-MIL significantly refines the coarse annotations, outperforming the state-of-the-art alternatives, even while learning from a single slide.
  arXiv  Detail & Related papers  (2021-09-22T15:06:06Z)
• Semi-supervised Medical Image Classification with Relation-driven Self-ensembling Model [71.80319052891817]
  We present a relation-driven semi-supervised framework for medical image classification. It exploits the unlabeled data by encouraging the prediction consistency of given input under perturbations. Our method outperforms many state-of-the-art semi-supervised learning methods on both single-label and multi-label image classification scenarios.
  arXiv  Detail & Related papers  (2020-05-15T06:57:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.