Spatial-aware Symmetric Alignment for Text-guided Medical Image Segmentation

• URL: http://arxiv.org/abs/2512.22981v1
• Date: Sun, 28 Dec 2025 16:02:42 GMT
• Title: Spatial-aware Symmetric Alignment for Text-guided Medical Image Segmentation
• Authors: Linglin Liao, Qichuan Geng, Yu Liu,
• Abstract summary: Text-guided Medical Image has shown considerable promise for medical image segmentation.<n>We propose the Spatial-aware Symmetric Alignment (SSA) framework to enhance the capacity of referring hybrid medical texts.<n>SSA achieves state-of-the-art (SOTA) performance, particularly in accurately segmenting lesions characterized by spatial constraints.
• Score: 7.514759533994352
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Text-guided Medical Image Segmentation has shown considerable promise for medical image segmentation, with rich clinical text serving as an effective supplement for scarce data. However, current methods have two key bottlenecks. On one hand, they struggle to process diagnostic and descriptive texts simultaneously, making it difficult to identify lesions and establish associations with image regions. On the other hand, existing approaches focus on lesions description and fail to capture positional constraints, leading to critical deviations. Specifically, with the text "in the left lower lung", the segmentation results may incorrectly cover both sides of the lung. To address the limitations, we propose the Spatial-aware Symmetric Alignment (SSA) framework to enhance the capacity of referring hybrid medical texts consisting of locational, descriptive, and diagnostic information. Specifically, we propose symmetric optimal transport alignment mechanism to strengthen the associations between image regions and multiple relevant expressions, which establishes bi-directional fine-grained multimodal correspondences. In addition, we devise a composite directional guidance strategy that explicitly introduces spatial constraints in the text by constructing region-level guidance masks. Extensive experiments on public benchmarks demonstrate that SSA achieves state-of-the-art (SOTA) performance, particularly in accurately segmenting lesions characterized by spatial relational constraints.

Related papers

• Structure Observation Driven Image-Text Contrastive Learning for Computed Tomography Report Generation [51.509572354327986]
  This work introduces a novel two-stage (structure- and report-learning) framework tailored for Computed Tomography Report Generation (CTRG)<n>In the first stage, a set of learnable structure-specific visual queries observe corresponding structures in a CT image. The resulting observation tokens are contrasted with structure-specific textual features extracted from the accompanying radiology report with a structure-wise image-text contrastive loss.<n>In the second stage, the visual structure queries are frozen and used to select the critical image patch embeddings depicting each anatomical structure, minimizing distractions from irrelevant areas while reducing memory consumption.
  arXiv  Detail & Related papers  (2026-03-05T07:07:07Z)
• TGC-Net: A Structure-Aware and Semantically-Aligned Framework for Text-Guided Medical Image Segmentation [56.09179939570486]
  We propose TGC-Net, a CLIP-based framework focusing on parameter-efficient, task-specific adaptations.<n>TGC-Net achieves state-of-the-art performance with substantially fewer trainable parameters, including notable Dice gains on challenging benchmarks.
  arXiv  Detail & Related papers  (2025-12-24T12:06:26Z)
• Self-Supervised Anatomical Consistency Learning for Vision-Grounded Medical Report Generation [61.350584471060756]
  Vision-grounded medical report generation aims to produce clinically accurate descriptions of medical images.<n>We propose Self-Supervised Anatomical Consistency Learning (SS-ACL) to align generated reports with corresponding anatomical regions.<n>SS-ACL constructs a hierarchical anatomical graph inspired by the invariant top-down inclusion structure of human anatomy.
  arXiv  Detail & Related papers  (2025-09-30T08:59:06Z)
• Progressive Local Alignment for Medical Multimodal Pre-training [24.56496333066882]
  We propose a contrastive learning-based approach for local alignment to establish meaningful word-pixel relationships.<n>PLAN effectively improves soft region recognition while suppressing noise interference.<n>PLAN surpasses state-of-the-art methods in phrase grounding, image-text retrieval, object detection, and zero-shot classification.
  arXiv  Detail & Related papers  (2025-02-25T10:13:13Z)
• Language-guided Medical Image Segmentation with Target-informed Multi-level Contrastive Alignments [7.9714765680840625]
  We propose a language-guided segmentation network with Target-informed Multi-level Contrastive Alignments (TMCA)<n>TMCA enables target-informed cross-modality alignments and fine-grained text guidance to bridge the pattern gaps in language-guided segmentation.
  arXiv  Detail & Related papers  (2024-12-18T06:19:03Z)
• HoliSDiP: Image Super-Resolution via Holistic Semantics and Diffusion Prior [62.04939047885834]
  We present HoliSDiP, a framework that leverages semantic segmentation to provide both precise textual and spatial guidance for Real-ISR.<n>Our method employs semantic labels as concise text prompts while introducing dense semantic guidance through segmentation masks and our proposed spatial-CLIP Map.
  arXiv  Detail & Related papers  (2024-11-27T15:22:44Z)
• A Multimodal Approach Combining Structural and Cross-domain Textual Guidance for Weakly Supervised OCT Segmentation [12.948027961485536]
  We propose a novel Weakly Supervised Semantic (WSSS) approach that integrates structural guidance with text-driven strategies to generate high-quality pseudo labels. Our method achieves state-of-the-art performance, highlighting its potential to improve diagnostic accuracy and efficiency in medical imaging.
  arXiv  Detail & Related papers  (2024-11-19T16:20:27Z)
• SGSeg: Enabling Text-free Inference in Language-guided Segmentation of Chest X-rays via Self-guidance [10.075820470715374]
  We propose a self-guided segmentation framework (SGSeg) that leverages language guidance for training (multi-modal) while enabling text-free inference (uni-modal) We exploit the critical location information of both pulmonary and pathological structures depicted in the text reports and introduce a novel localization-enhanced report generation (LERG) module to generate clinical reports for self-guidance. Our LERG integrates an object detector and a location-based attention aggregator, weakly-supervised by a location-aware pseudo-label extraction module.
  arXiv  Detail & Related papers  (2024-09-07T08:16:00Z)
• FlowSDF: Flow Matching for Medical Image Segmentation Using Distance Transforms [60.195642571004804]
  We introduce FlowSDF, an image-guided conditional flow matching framework, to represent an implicit distribution of segmentation masks.<n>Our framework enables accurate sampling of segmentation masks and the computation of relevant statistical measures.
  arXiv  Detail & Related papers  (2024-05-28T11:47:12Z)
• Cross-level Contrastive Learning and Consistency Constraint for Semi-supervised Medical Image Segmentation [46.678279106837294]
  We propose a cross-level constrastive learning scheme to enhance representation capacity for local features in semi-supervised medical image segmentation. With the help of the cross-level contrastive learning and consistency constraint, the unlabelled data can be effectively explored to improve segmentation performance.
  arXiv  Detail & Related papers  (2022-02-08T15:12:11Z)
• Unsupervised Bidirectional Cross-Modality Adaptation via Deeply Synergistic Image and Feature Alignment for Medical Image Segmentation [73.84166499988443]
  We present a novel unsupervised domain adaptation framework, named as Synergistic Image and Feature Alignment (SIFA) Our proposed SIFA conducts synergistic alignment of domains from both image and feature perspectives. Experimental results on two different tasks demonstrate that our SIFA method is effective in improving segmentation performance on unlabeled target images.
  arXiv  Detail & Related papers  (2020-02-06T13:49:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.