Context Matters: Support Set Selection and Failure Detection for In-Context Medical Image Segmentation

• URL: http://arxiv.org/abs/2608.05333v1
• Date: Wed, 05 Aug 2026 18:43:42 GMT
• Title: Context Matters: Support Set Selection and Failure Detection for In-Context Medical Image Segmentation
• Abstract summary: In-context learning adapts medical image segmentation models to unseen structures and modalities without retraining.<n>We investigate the support set as a controllable determinant of ICL reliability.
• Score: 7.942205885692331
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In-context learning (ICL) adapts medical image segmentation models to unseen structures and modalities without retraining by conditioning on a task-specific support set of image-mask exemplars. Because this support set is the model's only task-specific signal, its composition directly influences segmentation performance. In this work, we investigate the support set as a controllable determinant of ICL reliability. First, we compare random sampling against similarity-based selection, where exemplars are retrieved based on their visual similarity to the query image. Second, we train a transformer-based classifier to predict, from the query and support images alone, whether a segmentation will fall below a specified Intersection-over-Union (IoU) threshold. Using MultiverSeg with DINOv3 embeddings across four benchmarks and three imaging modalities, we show that similarity-based selection consistently matches or outperforms random sampling, with the largest gains at the smallest support set sizes. Furthermore, our classifier predicts segmentation failure above chance on all four benchmarks. Ultimately, these results demonstrate that the reliability of in-context segmentation can be both improved via informed support selection and anticipated before use, providing practical mechanisms for safer clinical deployment.

Related papers

• Active few-shot segmentation by reinforcing data selection [1.7289593940332797]
  Few-shot learning enables medical image segmentation models to adapt to new tasks using only a small number of labelled examples.<n>Support sets should capture relevant variation within the target domain and be informative for adaptation.<n>We propose a reinforcement learning framework for support-set selection in few-shot medical image segmentation.
  arXiv  Detail & Related papers  (2026-07-24T14:58:35Z)
• CSMCIR: CoT-Enhanced Symmetric Alignment with Memory Bank for Composed Image Retrieval [54.15776146365823]
  Composed Image Retrieval (CIR) enables users to search for target images using both a reference image and manipulation text.<n>We propose CSMCIR, a unified representation framework that achieves efficient query-target alignment through three synergistic components.
  arXiv  Detail & Related papers  (2026-01-07T09:21:38Z)
• Cycle Context Verification for In-Context Medical Image Segmentation [43.416111396585165]
  In-context learning (ICL) is emerging as a promising technique for achieving universal medical image segmentation.<n>In a clinical scenario, the scarcity of annotated medical images makes it challenging to select optimal in-context pairs.<n>We propose Cycle Context Verification (CCV), a novel framework that enhances ICL-based medical image segmentation.
  arXiv  Detail & Related papers  (2025-07-11T07:18:01Z)
• CONSIGN: Conformal Segmentation Informed by Spatial Groupings via Decomposition [0.0]
  Most machine learning-based image segmentation models produce pixel-wise confidence scores that represent the model's predicted probability for each class label at every pixel.<n>Conformal prediction provides a principled framework for transforming confidence scores into statistically valid uncertainty estimates.
  arXiv  Detail & Related papers  (2025-05-20T09:19:37Z)
• One-shot In-context Part Segmentation [97.77292483684877]
  We present the One-shot In-context Part (OIParts) framework to tackle the challenges of part segmentation.<n>Our framework offers a novel approach to part segmentation that is training-free, flexible, and data-efficient.<n>We have achieved remarkable segmentation performance across diverse object categories.
  arXiv  Detail & Related papers  (2025-03-03T03:50:54Z)
• FlowSDF: Flow Matching for Medical Image Segmentation Using Distance Transforms [60.195642571004804]
  We introduce FlowSDF, an image-guided conditional flow matching framework, to represent an implicit distribution of segmentation masks.<n>Our framework enables accurate sampling of segmentation masks and the computation of relevant statistical measures.
  arXiv  Detail & Related papers  (2024-05-28T11:47:12Z)
• Boosting Few-Shot Segmentation via Instance-Aware Data Augmentation and Local Consensus Guided Cross Attention [7.939095881813804]
  Few-shot segmentation aims to train a segmentation model that can fast adapt to a novel task for which only a few annotated images are provided. We introduce an instance-aware data augmentation (IDA) strategy that augments the support images based on the relative sizes of the target objects. The proposed IDA effectively increases the support set's diversity and promotes the distribution consistency between support and query images.
  arXiv  Detail & Related papers  (2024-01-18T10:29:10Z)
• Dense Affinity Matching for Few-Shot Segmentation [83.65203917246745]
  Few-Shot (FSS) aims to segment the novel class images with a few samples. We propose a dense affinity matching framework to exploit the support-query interaction. We show that our framework performs very competitively under different settings with only 0.68M parameters.
  arXiv  Detail & Related papers  (2023-07-17T12:27:15Z)
• Few-shot Medical Image Segmentation via Cross-Reference Transformer [3.2634122554914]
  Few-shot segmentation(FSS) has the potential to address these challenges by learning new categories from a small number of labeled samples. We propose a novel self-supervised few shot medical image segmentation network with Cross-Reference Transformer. Experimental results show that the proposed model achieves good results on both CT dataset and MRI dataset.
  arXiv  Detail & Related papers  (2023-04-19T13:05:18Z)
• Revisiting the Evaluation of Image Synthesis with GANs [55.72247435112475]
  This study presents an empirical investigation into the evaluation of synthesis performance, with generative adversarial networks (GANs) as a representative of generative models. In particular, we make in-depth analyses of various factors, including how to represent a data point in the representation space, how to calculate a fair distance using selected samples, and how many instances to use from each set.
  arXiv  Detail & Related papers  (2023-04-04T17:54:32Z)
• Few-Shot Segmentation via Cycle-Consistent Transformer [74.49307213431952]
  We focus on utilizing pixel-wise relationships between support and target images to facilitate the few-shot semantic segmentation task. We propose using a novel cycle-consistent attention mechanism to filter out possible harmful support features. Our proposed CyCTR leads to remarkable improvement compared to previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-06-04T07:57:48Z)
• Inter-class Discrepancy Alignment for Face Recognition [55.578063356210144]
  We propose a unified framework calledInter-class DiscrepancyAlignment(IDA) IDA-DAO is used to align the similarity scores considering the discrepancy between the images and its neighbors. IDA-SSE can provide convincing inter-class neighbors by introducing virtual candidate images generated with GAN.
  arXiv  Detail & Related papers  (2021-03-02T08:20:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.