ProSona: Prompt-Guided Personalization for Multi-Expert Medical Image Segmentation

• URL: http://arxiv.org/abs/2511.08046v1
• Date: Wed, 12 Nov 2025 01:36:38 GMT
• Title: ProSona: Prompt-Guided Personalization for Multi-Expert Medical Image Segmentation
• Authors: Aya Elgebaly, Nikolaos Delopoulos, Juliane Hörner-Rieber, Carolin Rippke, Sebastian Klüter, Luca Boldrini, Lorenzo Placidi, Riccardo Dal Bello, Nicolaus Andratschke, Michael Baumgartl, Claus Belka, Christopher Kurz, Guillaume Landry, Shadi Albarqouni,
• Abstract summary: We introduce ProSona, a framework that learns a continuous latent space of annotation styles, enabling controllable personalization via natural language prompts.<n>A probabilistic U-Net backbone captures diverse expert hypotheses, while a prompt-guided projection mechanism navigates this latent space to generate personalized segmentations.<n>Across the LIDC-IDRI lung nodule and multi-institutional prostate MRI datasets, ProSona reduces the Generalized Energy Distance by 17% and improves mean Dice by more than one point compared with DPersona.
• Score: 1.7251279593823998
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Automated medical image segmentation suffers from high inter-observer variability, particularly in tasks such as lung nodule delineation, where experts often disagree. Existing approaches either collapse this variability into a consensus mask or rely on separate model branches for each annotator. We introduce ProSona, a two-stage framework that learns a continuous latent space of annotation styles, enabling controllable personalization via natural language prompts. A probabilistic U-Net backbone captures diverse expert hypotheses, while a prompt-guided projection mechanism navigates this latent space to generate personalized segmentations. A multi-level contrastive objective aligns textual and visual representations, promoting disentangled and interpretable expert styles. Across the LIDC-IDRI lung nodule and multi-institutional prostate MRI datasets, ProSona reduces the Generalized Energy Distance by 17% and improves mean Dice by more than one point compared with DPersona. These results demonstrate that natural-language prompts can provide flexible, accurate, and interpretable control over personalized medical image segmentation. Our implementation is available online 1 .

Related papers

• ProGiDiff: Prompt-Guided Diffusion-Based Medical Image Segmentation [12.964514627034122]
  We propose a novel framework called ProGiDiff that leverages existing image generation models for medical image segmentation purposes.<n> Specifically, we propose a ControlNet-style conditioning mechanism with a custom encoder, suitable for image conditioning, to steer a pre-trained diffusion model to output segmentation masks.<n>Our experiment on organ segmentation from CT images demonstrates strong performance compared to previous methods and could greatly benefit from an expert-in-the-loop setting.
  arXiv  Detail & Related papers  (2026-01-22T15:56:21Z)
• Probabilistic Modeling of Multi-rater Medical Image Segmentation for Diversity and Personalization [47.42588216085903]
  We propose Probabilistic modeling of multi-rater medical image (ProSeg)<n>Our ProSeg achieves a new state-of-the-art performance, providing segmentation results that are both diverse and expert-personalized.
  arXiv  Detail & Related papers  (2025-11-30T05:53:39Z)
• DuPLUS: Dual-Prompt Vision-Language Framework for Universal Medical Image Segmentation and Prognosis [5.494301428436596]
  We introduce DuPLUS, a deep learning framework for efficient multi-modal medical image analysis.<n>DuPLUS introduces a novel vision-language framework that leverages hierarchical semantic prompts for fine-grained control over the analysis task.<n>For segmentation, DuPLUS is able to generalize across three imaging modalities, ten different various medical datasets, encompassing more than 30 organs and tumor types.
  arXiv  Detail & Related papers  (2025-10-03T20:01:00Z)
• UniSegDiff: Boosting Unified Lesion Segmentation via a Staged Diffusion Model [53.34835793648352]
  We propose UniSegDiff, a novel diffusion model framework for lesion segmentation.<n>UniSegDiff addresses lesion segmentation in a unified manner across multiple modalities and organs.<n> Comprehensive experimental results demonstrate that UniSegDiff significantly outperforms previous state-of-the-art (SOTA) approaches.
  arXiv  Detail & Related papers  (2025-07-24T12:33:10Z)
• DiffOSeg: Omni Medical Image Segmentation via Multi-Expert Collaboration Diffusion Model [12.103957886785926]
  variability remains a substantial challenge in medical image segmentation, stemming from ambiguous imaging boundaries and diverse clinical expertise.<n>We propose DiffOSeg, a two-stage diffusion-based framework, which aims to simultaneously achieve both consensus-driven and preference-driven segmentation.<n>Our model outperforms existing state-of-the-art methods across all evaluated metrics.
  arXiv  Detail & Related papers  (2025-07-17T12:57:27Z)
• Unlocking the Power of Spatial and Temporal Information in Medical Multimodal Pre-training [99.2891802841936]
  We introduce the Med-ST framework for fine-grained spatial and temporal modeling. For spatial modeling, Med-ST employs the Mixture of View Expert (MoVE) architecture to integrate different visual features from both frontal and lateral views. For temporal modeling, we propose a novel cross-modal bidirectional cycle consistency objective by forward mapping classification (FMC) and reverse mapping regression (RMR)
  arXiv  Detail & Related papers  (2024-05-30T03:15:09Z)
• Diversified and Personalized Multi-rater Medical Image Segmentation [43.47142636000329]
  We propose a two-stage framework named D-Persona (first Diversification and then Personalization). In Stage I, we exploit multiple given annotations to train a Probabilistic U-Net model, with a bound-constrained loss to improve the prediction diversity. In Stage II, we design multiple attention-based projection heads to adaptively query the corresponding expert prompts from the shared latent space, and then perform the personalized medical image segmentation.
  arXiv  Detail & Related papers  (2024-03-20T09:00:19Z)
• Annotator Consensus Prediction for Medical Image Segmentation with Diffusion Models [70.3497683558609]
  A major challenge in the segmentation of medical images is the large inter- and intra-observer variability in annotations provided by multiple experts. We propose a novel method for multi-expert prediction using diffusion models.
  arXiv  Detail & Related papers  (2023-06-15T10:01:05Z)
• Ambiguous Medical Image Segmentation using Diffusion Models [60.378180265885945]
  We introduce a single diffusion model-based approach that produces multiple plausible outputs by learning a distribution over group insights. Our proposed model generates a distribution of segmentation masks by leveraging the inherent sampling process of diffusion. Comprehensive results show that our proposed approach outperforms existing state-of-the-art ambiguous segmentation networks.
  arXiv  Detail & Related papers  (2023-04-10T17:58:22Z)
• Using Soft Labels to Model Uncertainty in Medical Image Segmentation [0.0]
  We propose a simple method to obtain soft labels from the annotations of multiple physicians. For each image, our method produces a single well-calibrated output that can be thresholded at multiple confidence levels. We evaluated our method on the MICCAI 2021 QUBIQ challenge, showing that it performs well across multiple medical image segmentation tasks.
  arXiv  Detail & Related papers  (2021-09-26T14:47:18Z)
• Weakly supervised multiple instance learning histopathological tumor segmentation [51.085268272912415]
  We propose a weakly supervised framework for whole slide imaging segmentation. We exploit a multiple instance learning scheme for training models. The proposed framework has been evaluated on multi-locations and multi-centric public data from The Cancer Genome Atlas and the PatchCamelyon dataset.
  arXiv  Detail & Related papers  (2020-04-10T13:12:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.