UniMedSeg: Unified In-Context Learning for Multi-Paradigm 2D/3D Medical Image Segmentation

• URL: http://arxiv.org/abs/2607.12896v1
• Date: Tue, 14 Jul 2026 15:34:46 GMT
• Title: UniMedSeg: Unified In-Context Learning for Multi-Paradigm 2D/3D Medical Image Segmentation
• Abstract summary: UniMedSeg is a Transformer-centric universal segmentation framework for medical images.<n>It maps visual examples, geometric interactions, language instructions, and 2D/3D images into a shared sequence space.<n>It achieves state-of-the-art performance across visual in-context, interactive, and language-guided segmentation without task-specific fine-tuning.
• Score: 44.30877351899412
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Medical image segmentation foundation models are expected to generalize across diverse clinical scenarios, yet existing universal methods remain fragmented by prompt paradigms and spatial dimensions. Visual in-context learning, interactive segmentation, and language-guided segmentation are typically handled by paradigm-specific models, while 2D and 3D images are also modeled separately. Such isolation prevents heterogeneous annotations and data from being jointly absorbed by a single scalable model and limits cross-paradigm knowledge transfer. To address this bottleneck, we propose UniMedSeg, a Transformer-centric universal segmentation framework that maps visual examples, geometric interactions, language instructions, and 2D/3D images into a shared sequence space, enabling heterogeneous medical supervision to be jointly learned through a unified in-context interface without prompt- or dimension-specific branches. To overcome the long-sequence memory bottleneck caused by visual contexts, we introduce Decoupled Split Attention, which reduces attention complexity to linear while preserving hardware-friendly computation and focused context-target interaction. Extensively trained and evaluated on a large corpus curated from 27 public datasets, UniMedSeg achieves state-of-the-art performance across visual in-context, interactive, and language-guided segmentation without task-specific fine-tuning, demonstrating strong generalization on diverse held-out tasks. The code and model weights are publicly available at https://github.com/Lii1228/UniMedSeg

Related papers

• CoInS-Net: A Continuous Position-Aware Network for Joint Medical Image Interpolation and Segmentation [8.415835019440769]
  We propose a continuous position-continuity interaction network, termed CoInS-Net, for joint frameaware and lesion segmentation.<n>A prototype-based task mutual interaction module lets the segmentation and branches exchange global structure.<n>A multiscale task-cooperative decoder further separates each scale into shared and task-specific components, so the two tasks reinforce common anatomy.
  arXiv  Detail & Related papers  (2026-08-10T10:19:16Z)
• SLIP: Segmentation with Low-latency Interactive Prompting for 3D Medical Images [58.57521567704563]
  We present SLIP, an end-to-end trainable framework for interactive 3D medical image segmentation.<n>By separating image representation from interactive reasoning, SLIP substantially reduces interaction latency.<n>SLIP achieves SOTA interactive segmentation performance across 13 public datasets.
  arXiv  Detail & Related papers  (2026-07-24T14:12:02Z)
• Concept-to-Pixel: Prompt-Free Universal Medical Image Segmentation [2.5026850988034797]
  Concept-to-Pixel (C2P) is a novel prompt-free universal segmentation framework.<n>C2P separates anatomical knowledge into two components: Geometric and Semantic representations.
  arXiv  Detail & Related papers  (2026-03-18T14:13:22Z)
• Unlocking the Power of Spatial and Temporal Information in Medical Multimodal Pre-training [99.2891802841936]
  We introduce the Med-ST framework for fine-grained spatial and temporal modeling. For spatial modeling, Med-ST employs the Mixture of View Expert (MoVE) architecture to integrate different visual features from both frontal and lateral views. For temporal modeling, we propose a novel cross-modal bidirectional cycle consistency objective by forward mapping classification (FMC) and reverse mapping regression (RMR)
  arXiv  Detail & Related papers  (2024-05-30T03:15:09Z)
• Dual-scale Enhanced and Cross-generative Consistency Learning for Semi-supervised Medical Image Segmentation [49.57907601086494]
  Medical image segmentation plays a crucial role in computer-aided diagnosis. We propose a novel Dual-scale Enhanced and Cross-generative consistency learning framework for semi-supervised medical image (DEC-Seg)
  arXiv  Detail & Related papers  (2023-12-26T12:56:31Z)
• Segment Everything Everywhere All at Once [124.90835636901096]
  We present SEEM, a promptable and interactive model for segmenting everything everywhere all at once in an image. We propose a novel decoding mechanism that enables diverse prompting for all types of segmentation tasks. We conduct a comprehensive empirical study to validate the effectiveness of SEEM across diverse segmentation tasks.
  arXiv  Detail & Related papers  (2023-04-13T17:59:40Z)
• Implicit Anatomical Rendering for Medical Image Segmentation with Stochastic Experts [11.007092387379078]
  We propose MORSE, a generic implicit neural rendering framework designed at an anatomical level to assist learning in medical image segmentation. Our approach is to formulate medical image segmentation as a rendering problem in an end-to-end manner. Our experiments demonstrate that MORSE can work well with different medical segmentation backbones.
  arXiv  Detail & Related papers  (2023-04-06T16:44:03Z)
• CUTS: A Deep Learning and Topological Framework for Multigranular Unsupervised Medical Image Segmentation [8.307551496968156]
  We present CUTS, an unsupervised deep learning framework for medical image segmentation. For each image, it produces an embedding map via intra-image contrastive learning and local patch reconstruction. CUTS yields a series of coarse-to-fine-grained segmentations that highlight features at various granularities.
  arXiv  Detail & Related papers  (2022-09-23T01:09:06Z)
• Open-world Semantic Segmentation via Contrasting and Clustering Vision-Language Embedding [95.78002228538841]
  We propose a new open-world semantic segmentation pipeline that makes the first attempt to learn to segment semantic objects of various open-world categories without any efforts on dense annotations. Our method can directly segment objects of arbitrary categories, outperforming zero-shot segmentation methods that require data labeling on three benchmark datasets.
  arXiv  Detail & Related papers  (2022-07-18T09:20:04Z)
• Two-Stream Graph Convolutional Network for Intra-oral Scanner Image Segmentation [133.02190910009384]
  We propose a two-stream graph convolutional network (i.e., TSGCN) to handle inter-view confusion between different raw attributes. Our TSGCN significantly outperforms state-of-the-art methods in 3D tooth (surface) segmentation.
  arXiv  Detail & Related papers  (2022-04-19T10:41:09Z)
• Momentum Contrastive Voxel-wise Representation Learning for Semi-supervised Volumetric Medical Image Segmentation [2.3322477552758234]
  We present a novel Contrastive Voxel-wise Representation (CVRL) method with geometric constraints to learn global-local visual representations for medical image segmentation. Our framework can effectively learn global and local features by capturing 3D spatial context and rich anatomical information.
  arXiv  Detail & Related papers  (2021-05-14T20:27:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.