From Adaptation to Generalization: Adaptive Visual Prompting for Medical Image Segmentation

• URL: http://arxiv.org/abs/2604.17455v1
• Date: Sun, 19 Apr 2026 14:14:10 GMT
• Title: From Adaptation to Generalization: Adaptive Visual Prompting for Medical Image Segmentation
• Abstract summary: We propose an Adaptive Prompt EXtraction framework that retrieves input-specific prompts from a learnable prompt memory.<n>The memory stores diverse, domain-discriminative prompt representations and is queried via domain features extracted from the Fourier spectrum.<n>Experiments on two medical segmentation tasks demonstrate that APEX significantly improves generalization across both seen and unseen domains.
• Score: 58.93075548229873
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Visual prompting has emerged as a powerful method for adapting pre-trained models to new domains without updating model parameters. However, existing prompting methods typically optimize a single prompt per domain and apply it uniformly to all inputs, limiting their ability to generalize under intra and inter-domain variability, which is especially critical in the medical field. To address this, we propose APEX, an Adaptive Prompt EXtraction framework that retrieves input-specific prompts from a learnable prompt memory. The memory stores diverse, domain-discriminative prompt representations and is queried via domain features extracted from the Fourier spectrum. To learn robust and discriminative domain features, we introduce a novel Low-Frequency Feature Contrastive (LFC) learning framework that clusters representations from the same domain while separating those from different domains. Extensive experiments on two medical segmentation tasks demonstrate that APEX significantly improves generalization across both seen and unseen domains. Furthermore, it complements any existing backbones and consistently enhances performance, confirming its effectiveness as a plug-and-play prompting solution in medical fields. The code is available at https://github.com/cetinkayaevren/apex/

Related papers

• QdaVPR: A novel query-based domain-agnostic model for visual place recognition [2.9951269291329905]
  Visual place recognition (VPR) aims at predicting the location of an image based solely on its visual features.<n>This paper proposes a novel query-based domain-agnostic VPR model called QdaVPR.<n>It achieves state-of-the-art performance on multiple VPR benchmarks with significant domain variations.
  arXiv  Detail & Related papers  (2026-03-08T02:01:20Z)
• HyDA: Hypernetworks for Test Time Domain Adaptation in Medical Imaging Analysis [4.450536872346658]
  We introduce HyDA, a novel hypernetwork framework that leverages domain characteristics rather than suppressing them.<n>Specifically, HyDA learns implicit domain representations and uses them to adjust model parameters on-the-fly.<n>We validate HyDA on two clinically relevant applications - MRI brain age prediction and chest X-ray pathology classification.
  arXiv  Detail & Related papers  (2025-03-06T21:17:40Z)
• Prompting Segment Anything Model with Domain-Adaptive Prototype for Generalizable Medical Image Segmentation [49.5901368256326]
  We propose a novel Domain-Adaptive Prompt framework for fine-tuning the Segment Anything Model (termed as DAPSAM) in segmenting medical images. Our DAPSAM achieves state-of-the-art performance on two medical image segmentation tasks with different modalities.
  arXiv  Detail & Related papers  (2024-09-19T07:28:33Z)
• Role Prompting Guided Domain Adaptation with General Capability Preserve for Large Language Models [55.51408151807268]
  When tailored to specific domains, Large Language Models (LLMs) tend to experience catastrophic forgetting. crafting a versatile model for multiple domains simultaneously often results in a decline in overall performance. We present the RolE Prompting Guided Multi-Domain Adaptation (REGA) strategy.
  arXiv  Detail & Related papers  (2024-03-05T08:22:41Z)
• Memory-Efficient Prompt Tuning for Incremental Histopathology Classification [69.46798702300042]
  We present a memory-efficient prompt tuning framework to cultivate model generalization potential in economical memory cost. We have extensively evaluated our framework with two histopathology tasks, i.e., breast cancer metastasis classification and epithelium-stroma tissue classification.
  arXiv  Detail & Related papers  (2024-01-22T03:24:45Z)
• Domain-Controlled Prompt Learning [49.45309818782329]
  Existing prompt learning methods often lack domain-awareness or domain-transfer mechanisms. We propose a textbfDomain-Controlled Prompt Learning for the specific domains. Our method achieves state-of-the-art performance in specific domain image recognition datasets.
  arXiv  Detail & Related papers  (2023-09-30T02:59:49Z)
• Prompting Diffusion Representations for Cross-Domain Semantic Segmentation [101.04326113360342]
  diffusion-pretraining achieves extraordinary domain generalization results for semantic segmentation. We introduce a scene prompt and a prompt randomization strategy to help further disentangle the domain-invariant information when training the segmentation head.
  arXiv  Detail & Related papers  (2023-07-05T09:28:25Z)
• Learning Domain Invariant Prompt for Vision-Language Models [31.581652862478965]
  We propose a novel prompt learning paradigm that directly generates emphdomain invariant prompt that can be generalized to unseen domains, called MetaPrompt. Our method consistently and significantly outperforms existing methods.
  arXiv  Detail & Related papers  (2022-12-08T11:23:24Z)
• Domain Adaptation via Prompt Learning [39.97105851723885]
  Unsupervised domain adaption (UDA) aims to adapt models learned from a well-annotated source domain to a target domain. We introduce a novel prompt learning paradigm for UDA, named Domain Adaptation via Prompt Learning (DAPL)
  arXiv  Detail & Related papers  (2022-02-14T13:25:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.