DeferredSeg: A Multi-Expert Deferral Framework for Trustworthy Medical Image Segmentation

• URL: http://arxiv.org/abs/2604.12411v1
• Date: Tue, 14 Apr 2026 07:50:31 GMT
• Title: DeferredSeg: A Multi-Expert Deferral Framework for Trustworthy Medical Image Segmentation
• Abstract summary: We introduce DeferredSeg, a deferral-aware segmentation framework that defers predictions to human experts in specific regions.<n>We evaluate DeferredSeg on three challenging medical datasets using MedSAM and CENet as the base segmentor for fair comparison.<n> Experimental results show that DeferredSeg consistently outperforms the baseline.
• Score: 66.24948946364329
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Segmentation models based on deep neural networks demonstrate strong generalization for medical image segmentation. However, they often exhibit overconfidence or underconfidence, leading to unreliable confidence scores for segmentation masks, especially in ambiguous regions. This undermines the trustworthiness required for clinical deployment. Motivated by the learning-to-defer (L2D) paradigm, we introduce DeferredSeg, a deferral-aware segmentation framework, i.e., a Human--AI collaboration system that determines whether to defer predictions to human experts in specific regions. DeferredSeg extends the base segmentor with an aggregated deferral predictor and additional routing channels that dynamically route each pixel to either the base segmentor or a human expert. To train this routing efficiently, we introduce a pixel-wise surrogate collaboration loss that supervises deferral decisions. In addition, to preserve spatial coherence within deferral regions, we propose a spatial-coherence loss that enforces smooth deferral masks, thereby enhancing reliability. Beyond single-expert deferral, we further extend the framework to a multi-expert setting by introducing multiple discrepancy experts for collaborative decision-making. To prevent overloading or underutilizing individual experts, we further design a load-balancing penalty that evenly distributes workload across expert branches. We evaluate DeferredSeg on three challenging medical datasets using MedSAM and CENet as the base segmentor for fair comparison. Experimental results show that DeferredSeg consistently outperforms the baseline, demonstrating its effectiveness for trustworthy dense medical segmentation. Moreover, the proposed framework is model-agnostic and can be readily applied to other segmentation architectures.

Related papers

• Uncertainty-aware Cross-training for Semi-supervised Medical Image Segmentation [45.96892342675963]
  We propose an Uncertainty-aware Cross-training framework for semi-supervised medical image (UC-Seg)<n>Our method achieves superior segmentation accuracy and generalization performance compared to other state-of-the-art semi-supervised methods.
  arXiv  Detail & Related papers  (2025-08-12T15:28:10Z)
• DiffOSeg: Omni Medical Image Segmentation via Multi-Expert Collaboration Diffusion Model [12.103957886785926]
  variability remains a substantial challenge in medical image segmentation, stemming from ambiguous imaging boundaries and diverse clinical expertise.<n>We propose DiffOSeg, a two-stage diffusion-based framework, which aims to simultaneously achieve both consensus-driven and preference-driven segmentation.<n>Our model outperforms existing state-of-the-art methods across all evaluated metrics.
  arXiv  Detail & Related papers  (2025-07-17T12:57:27Z)
• MAMBO-NET: Multi-Causal Aware Modeling Backdoor-Intervention Optimization for Medical Image Segmentation Network [51.68708264694361]
  Confusion factors can affect medical images, such as complex anatomical variations and imaging modality limitations.<n>We propose a multi-causal aware modeling backdoor-intervention optimization network for medical image segmentation.<n>Our method significantly reduces the influence of confusion factors, leading to enhanced segmentation accuracy.
  arXiv  Detail & Related papers  (2025-05-28T01:40:10Z)
• Inter-Rater Uncertainty Quantification in Medical Image Segmentation via Rater-Specific Bayesian Neural Networks [7.642026462053574]
  We introduce a novel Bayesian neural network-based architecture to estimate inter-rater uncertainty in medical image segmentation. Firstly, we introduce a one-encoder-multi-decoder architecture specifically tailored for uncertainty estimation. Secondly, we propose Bayesian modeling for the new architecture, allowing efficient capture of the inter-rater distribution.
  arXiv  Detail & Related papers  (2023-06-28T20:52:51Z)
• Ambiguous Medical Image Segmentation using Diffusion Models [60.378180265885945]
  We introduce a single diffusion model-based approach that produces multiple plausible outputs by learning a distribution over group insights. Our proposed model generates a distribution of segmentation masks by leveraging the inherent sampling process of diffusion. Comprehensive results show that our proposed approach outperforms existing state-of-the-art ambiguous segmentation networks.
  arXiv  Detail & Related papers  (2023-04-10T17:58:22Z)
• Lung Nodule Segmentation and Uncertain Region Prediction with an Uncertainty-Aware Attention Mechanism [30.298653876400003]
  Radiologists possess diverse training and clinical experiences, leading to variations in the segmentation of lung nodules. Conventional methods typically select a single annotation as the learning target or attempt to learn a latent space comprising multiple annotations. We propose an Uncertainty-Aware Attention Mechanism (UAAM) that utilizes consensus and disagreements among multiple annotations to facilitate better segmentation.
  arXiv  Detail & Related papers  (2023-03-15T07:31:55Z)
• Reliable Joint Segmentation of Retinal Edema Lesions in OCT Images [55.83984261827332]
  In this paper, we propose a novel reliable multi-scale wavelet-enhanced transformer network. We develop a novel segmentation backbone that integrates a wavelet-enhanced feature extractor network and a multi-scale transformer module. Our proposed method achieves better segmentation accuracy with a high degree of reliability as compared to other state-of-the-art segmentation approaches.
  arXiv  Detail & Related papers  (2022-12-01T07:32:56Z)
• Inconsistency-aware Uncertainty Estimation for Semi-supervised Medical Image Segmentation [92.9634065964963]
  We present a new semi-supervised segmentation model, namely, conservative-radical network (CoraNet) based on our uncertainty estimation and separate self-training strategy. Compared with the current state of the art, our CoraNet has demonstrated superior performance.
  arXiv  Detail & Related papers  (2021-10-17T08:49:33Z)
• Few-shot Medical Image Segmentation using a Global Correlation Network with Discriminative Embedding [60.89561661441736]
  We propose a novel method for few-shot medical image segmentation. We construct our few-shot image segmentor using a deep convolutional network trained episodically. We enhance discriminability of deep embedding to encourage clustering of the feature domains of the same class.
  arXiv  Detail & Related papers  (2020-12-10T04:01:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.