Region-Grounded Vision-Language Learning for Detection-Guided Mammographic Lesion Classification

• URL: http://arxiv.org/abs/2607.15615v1
• Date: Fri, 17 Jul 2026 04:27:44 GMT
• Title: Region-Grounded Vision-Language Learning for Detection-Guided Mammographic Lesion Classification
• Abstract summary: We propose a novel region-grounded vision-language learning method for detection-guided mammographic lesion classification.<n>To mitigate semantic collapse and background bias in low-vocabulary settings, we introduce a multi-component objective incorporating positive alignment, fine-grained semantic hard negatives, and background suppression.<n>Experiments on two independent datasets, CBIS-DDSM and VinDr-Mammo, show superior performance of our method compared to related methods under in-domain, cross-dataset, and transfer learning settings.
• Score: 2.9370716053964916
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-language models trained with contrastive objectives have shown promise in medical image analysis. However, conventional global image-text alignment is ill-suited for mammography, where diagnostically relevant lesions are spatially localized and occupy only a small fraction of the image. Subtle morphological cues critical for malignancy assessment can be diluted when representations are learned at the whole-image level. In this work, we propose a novel region-grounded vision-language learning method for detection-guided mammographic lesion classification. The method mirrors radiologists' diagnostic paradigm. First, a region-text contrastive pretraining stage aligns lesion-specific features with structured clinical descriptors derived from radiology metadata. To mitigate semantic collapse and background bias in low-vocabulary settings, we introduce a multi-component objective incorporating positive alignment, fine-grained semantic hard negatives, and background suppression. Second, an auxiliary lesion detection head is jointly optimized with contrastive classification to preserve spatial sensitivity and enable localization-aware malignancy classification. Extensive experiments on two independent datasets, CBIS-DDSM and VinDr-Mammo, show superior performance of our method compared to related methods under in-domain, cross-dataset, and transfer learning settings.

Related papers

• Reconstruction-Shift Discrimination via Mask-Guided Latent Diffusion for Medical Anomaly Detection [58.92871907037662]
  Unsupervised medical anomaly detection learns normal anatomical patterns from healthy training images and identifies deviations at test time.<n>Reconstruction-based and diffusion-based methods commonly use the difference between an input image and its reconstruction as anomaly evidence.<n>We propose discriminative mask-guided diffusion (DMD), a medical anomaly detection framework that complements residual-based localization with reconstruction-shift discrimination.
  arXiv  Detail & Related papers  (2026-08-01T04:56:39Z)
• Multi-View Synergistic Learning with Vision-Language Adaption for Low-Resource Biomedical Image Classification [59.24009931000134]
  Multi-View Synergistic Learning (MVSL) is a unified framework that addresses adaptation paradigms, representation granularity, and disease semantic relationships.<n>MVSL decouples the adaptation of visual and textual encoders to respect their distinct representational characteristics.<n>It further introduces multi-granularity contrastive learning to explicitly model both global image semantics and localized lesion-level evidence.<n>MVSL consistently outperforms state-of-the-art methods in few-shot and zero-shot classification settings.
  arXiv  Detail & Related papers  (2026-04-27T02:41:27Z)
• Plasticine: A Traceable Diffusion Model for Medical Image Translation [79.39689106440389]
  We propose Plasticine, to the best of our knowledge, the first end-to-end image-to-image translation framework explicitly designed with traceability as a core objective.<n>Our method combines intensity translation and spatial transformation within a denoising diffusion framework.<n>This design enables the generation of synthetic images with interpretable intensity transitions and spatially coherent deformations, supporting pixel-wise traceability throughout the translation process.
  arXiv  Detail & Related papers  (2025-12-20T18:01:57Z)
• From Classification to Cross-Modal Understanding: Leveraging Vision-Language Models for Fine-Grained Renal Pathology [9.268389327736735]
  We model fine-grained glomerular subtyping as a clinically realistic few-shot problem.<n>We evaluate both pathology-specialized and general-purpose vision-language models under this setting.
  arXiv  Detail & Related papers  (2025-11-15T01:44:11Z)
• Seeing the Trees for the Forest: Rethinking Weakly-Supervised Medical Visual Grounding [45.248265283405004]
  Current models struggle to associate textual descriptions with disease regions due to inefficient attention mechanisms and a lack of fine-grained token representations.<n>We introduce Disease-Aware Prompting (DAP), which uses the explainability map of a VLM to identify the appropriate image features.<n>DAP improves visual grounding accuracy by 20.74% compared to state-of-the-art methods across three major chest X-ray datasets.
  arXiv  Detail & Related papers  (2025-05-21T05:16:45Z)
• Meta-Entity Driven Triplet Mining for Aligning Medical Vision-Language Models [9.76070837929117]
  Existing alignment methods prioritize separation between disease classes over segregation of fine-grained pathology attributes.<n>Here, we propose MedTrim, a novel method that enhances image-text alignment through multimodal triplet learning.<n>Our demonstrations indicate that MedTrim improves performance in downstream retrieval and classification tasks compared to state-of-the-art alignment methods.
  arXiv  Detail & Related papers  (2025-04-22T14:17:51Z)
• Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images [68.42215385041114]
  This paper introduces a novel lightweight multi-level adaptation and comparison framework to repurpose the CLIP model for medical anomaly detection. Our approach integrates multiple residual adapters into the pre-trained visual encoder, enabling a stepwise enhancement of visual features across different levels. Our experiments on medical anomaly detection benchmarks demonstrate that our method significantly surpasses current state-of-the-art models.
  arXiv  Detail & Related papers  (2024-03-19T09:28:19Z)
• A2V: A Semi-Supervised Domain Adaptation Framework for Brain Vessel Segmentation via Two-Phase Training Angiography-to-Venography Translation [4.452428104996953]
  We present a semi-supervised domain adaptation framework for brain vessel segmentation from different image modalities. By relying on annotated angiographies and a limited number of annotated venographies, our framework accomplishes image-to-image translation and semantic segmentation.
  arXiv  Detail & Related papers  (2023-09-12T09:12:37Z)
• Significantly improving zero-shot X-ray pathology classification via fine-tuning pre-trained image-text encoders [50.689585476660554]
  We propose a new fine-tuning strategy that includes positive-pair loss relaxation and random sentence sampling. Our approach consistently improves overall zero-shot pathology classification across four chest X-ray datasets and three pre-trained models.
  arXiv  Detail & Related papers  (2022-12-14T06:04:18Z)
• Learning Discriminative Representation via Metric Learning for Imbalanced Medical Image Classification [52.94051907952536]
  We propose embedding metric learning into the first stage of the two-stage framework specially to help the feature extractor learn to extract more discriminative feature representations. Experiments mainly on three medical image datasets show that the proposed approach consistently outperforms existing onestage and two-stage approaches.
  arXiv  Detail & Related papers  (2022-07-14T14:57:01Z)
• Segmentation-Renormalized Deep Feature Modulation for Unpaired Image Harmonization [0.43012765978447565]
  Cycle-consistent Generative Adversarial Networks have been used to harmonize image sets between a source and target domain. These methods are prone to instability, contrast inversion, intractable manipulation of pathology, and steganographic mappings which limit their reliable adoption in real-world medical imaging. We propose a segmentation-renormalized image translation framework to reduce inter-scanner harmonization while preserving anatomical layout.
  arXiv  Detail & Related papers  (2021-02-11T23:53:51Z)
• Breast Cancer Histopathology Image Classification and Localization using Multiple Instance Learning [2.4178424543973267]
  Computer-aided pathology to analyze microscopic histopathology images for diagnosis can bring the cost and delays of diagnosis down. Deep learning in histopathology has attracted attention over the last decade of achieving state-of-the-art performance in classification and localization tasks. We present classification and localization results on two publicly available BreakHIS and BACH dataset.
  arXiv  Detail & Related papers  (2020-02-16T10:29:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.