Developing Foundation Models for Universal Segmentation from 3D Whole-Body Positron Emission Tomography

• URL: http://arxiv.org/abs/2603.11627v1
• Date: Thu, 12 Mar 2026 07:41:28 GMT
• Title: Developing Foundation Models for Universal Segmentation from 3D Whole-Body Positron Emission Tomography
• Abstract summary: We build the largest and most comprehensive PET dataset to date, comprising 11041 3D whole-body PET scans with 59831 segmentation masks for model development.<n>We present SegAnyPET, an innovative foundational model with general-purpose applicability to diverse segmentation tasks.
• Score: 36.14746684754266
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Positron emission tomography (PET) is a key nuclear medicine imaging modality that visualizes radiotracer distributions to quantify in vivo physiological and metabolic processes, playing an irreplaceable role in disease management. Despite its clinical importance, the development of deep learning models for quantitative PET image analysis remains severely limited, driven by both the inherent segmentation challenge from PET's paucity of anatomical contrast and the high costs of data acquisition and annotation. To bridge this gap, we develop generalist foundational models for universal segmentation from 3D whole-body PET imaging. We first build the largest and most comprehensive PET dataset to date, comprising 11041 3D whole-body PET scans with 59831 segmentation masks for model development. Based on this dataset, we present SegAnyPET, an innovative foundational model with general-purpose applicability to diverse segmentation tasks. Built on a 3D architecture with a prompt engineering strategy for mask generation, SegAnyPET enables universal and scalable organ and lesion segmentation, supports efficient human correction with minimal effort, and enables a clinical human-in-the-loop workflow. Extensive evaluations on multi-center, multi-tracer, multi-disease datasets demonstrate that SegAnyPET achieves strong zero-shot performance across a wide range of segmentation tasks, highlighting its potential to advance the clinical applications of molecular imaging.

Related papers

• Unveiling and Bridging the Functional Perception Gap in MLLMs: Atomic Visual Alignment and Hierarchical Evaluation via PET-Bench [48.60251555171943]
  Multimodal Large Language Models (MLLMs) have demonstrated remarkable proficiency in tasks such as abnormality detection and report generation for anatomical modalities.<n>In this work, we quantify a fundamental functional perception gap: the inability of current vision encoders to decode functional tracer biodistribution independent of morphological priors.<n>We introduce PET-Bench, the first large-scale functional imaging benchmark comprising 52,308 hierarchical QA pairs from 9,732 multi-site, multi-tracer PET studies.<n>Our results demonstrate that AVA effectively bridges the perception gap, transforming CoT from a source of hallucination into a robust inference tool and improving diagnostic
  arXiv  Detail & Related papers  (2026-01-06T05:58:50Z)
• PETAR: Localized Findings Generation with Mask-Aware Vision-Language Modeling for PET Automated Reporting [10.800541081358132]
  We introduce a large-scale dataset comprising over 11,000 lesion-level descriptions paired with 3D segmentations from more than 5,000 PET/CT exams.<n>Building upon this dataset, we propose PETAR-4B, a 3D mask-aware vision-language model that integrates PET, CT, and lesion contours for spatially grounded report generation.
  arXiv  Detail & Related papers  (2025-10-31T17:49:01Z)
• Supervised Diffusion-Model-Based PET Image Reconstruction [44.89560992517543]
  Diffusion models (DMs) have been introduced as a regularizing prior for PET image reconstruction.<n>We propose a supervised DM-based algorithm for PET reconstruction.<n>Our method enforces the non-negativity of PET's Poisson likelihood model and accommodates the wide intensity range of PET images.
  arXiv  Detail & Related papers  (2025-06-30T16:39:50Z)
• Personalized MR-Informed Diffusion Models for 3D PET Image Reconstruction [40.722159771726375]
  We propose a simple method for generating subject-specific PET images from a dataset of PET-MR scans.<n>The images we synthesize retain information from the subject's MR scan, leading to higher resolution and the retention of anatomical features.<n>With simulated and real [$18$F]FDG datasets, we show that pre-training a personalized diffusion model with subject-specific "pseudo-PET" images improves reconstruction accuracy with low-count data.
  arXiv  Detail & Related papers  (2025-06-04T10:24:14Z)
• SegAnyPET: Universal Promptable Segmentation from Positron Emission Tomography Images [21.883098685700666]
  We develop SegAnyPET, a modality-specific 3D foundation model for universal promptable segmentation from PET images.<n> Experimental results demonstrate that SegAnyPET can segment seen and unseen target organs using only one or a few prompt points.
  arXiv  Detail & Related papers  (2025-02-20T08:17:13Z)
• AutoPET III Challenge: Tumor Lesion Segmentation using ResEnc-Model Ensemble [1.3467243219009812]
  We trained a 3D Residual encoder U-Net within the no new U-Net framework to generalize the performance of automatic lesion segmentation. We leveraged test-time augmentations and other post-processing techniques to enhance tumor lesion segmentation. Our team currently hold the top position in the Auto-PET III challenge and outperformed the challenge baseline model in the preliminary test set with Dice score of 0.9627.
  arXiv  Detail & Related papers  (2024-09-19T20:18:39Z)
• Autopet III challenge: Incorporating anatomical knowledge into nnUNet for lesion segmentation in PET/CT [4.376648893167674]
  The autoPET III Challenge focuses on advancing automated segmentation of tumor lesions in PET/CT images. We developed a classifier that identifies the tracer of the given PET/CT based on the Maximum Intensity Projection of the PET scan. Our final submission achieves cross-validation Dice scores of 76.90% and 61.33% for the publicly available FDG and PSMA datasets.
  arXiv  Detail & Related papers  (2024-09-18T17:16:57Z)
• From FDG to PSMA: A Hitchhiker's Guide to Multitracer, Multicenter Lesion Segmentation in PET/CT Imaging [0.9384264274298444]
  We present our solution for the autoPET III challenge, targeting multitracer, multicenter generalization using the nnU-Net framework with the ResEncL architecture. Key techniques include misalignment data augmentation and multi-modal pretraining across CT, MR, and PET datasets. Compared to the default nnU-Net, which achieved a Dice score of 57.61, our model significantly improved performance with a Dice score of 68.40, alongside a reduction in false positive (FPvol: 7.82) and false negative (FNvol: 10.35) volumes.
  arXiv  Detail & Related papers  (2024-09-14T16:39:17Z)
• Contrastive Diffusion Model with Auxiliary Guidance for Coarse-to-Fine PET Reconstruction [62.29541106695824]
  This paper presents a coarse-to-fine PET reconstruction framework that consists of a coarse prediction module (CPM) and an iterative refinement module (IRM) By delegating most of the computational overhead to the CPM, the overall sampling speed of our method can be significantly improved. Two additional strategies, i.e., an auxiliary guidance strategy and a contrastive diffusion strategy, are proposed and integrated into the reconstruction process.
  arXiv  Detail & Related papers  (2023-08-20T04:10:36Z)
• A Data-scalable Transformer for Medical Image Segmentation: Architecture, Model Efficiency, and Benchmark [45.543140413399506]
  MedFormer is a data-scalable Transformer designed for generalizable 3D medical image segmentation. Our approach incorporates three key elements: a desirable inductive bias, hierarchical modeling with linear-complexity attention, and multi-scale feature fusion.
  arXiv  Detail & Related papers  (2022-02-28T22:59:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.