An Open Multi-Center Whole-Body FDG PET/CT Foundation Model for Tumor Segmentation

• URL: http://arxiv.org/abs/2605.21835v1
• Date: Wed, 20 May 2026 23:59:50 GMT
• Title: An Open Multi-Center Whole-Body FDG PET/CT Foundation Model for Tumor Segmentation
• Abstract summary: We present an open-source, multi-center, whole-body FDG PET/CT foundation model utilizing 4,997 harmonized scans from four public datasets.<n>Our framework employs hierarchical UNet-shaped backbones with early channel-wise concatenation, enabling anatomical and metabolic features to interact.<n>It provides a robust, open-source basis for advancing automated oncologic imaging.
• Score: 9.922425508121012
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The synergistic interpretation of anatomical information from computed tomography (CT) and metabolic information from positron emission tomography (PET) is important to oncologic imaging. However, existing deep learning methods for PET/CT remain largely task-specific, are often trained on single-center cohorts, or adopt dual-branch fusion schemes that delay cross-modal interaction and underutilize early spatial correspondence between PET and CT. To address these limitations, we present an open-source, multi-center, whole-body FDG PET/CT foundation model utilizing 4,997 harmonized scans from four public datasets. Our framework employs hierarchical UNet-shaped backbones with early channel-wise concatenation, enabling anatomical and metabolic features to interact from the first embedding layer onward. We further introduce a masked autoencoding objective based on zero-mean imputation, combined with a weighted global reconstruction loss. This design avoids non-physical intensity discontinuities at masked-region boundaries that arise from learnable mask tokens. On downstream AutoPET lesion segmentation, the proposed models demonstrate strong label efficiency: with only 10\% of the labeled training data, they achieve performance comparable to models trained from scratch on the full dataset. Under extreme 5-shot linear probing, joint PET/CT pretraining also achieves higher Dice scores than separated-modality pretraining. This multi-center foundation model demonstrates label efficiency and cross-modality representation learning for PET/CT tumor segmentation. It provides a robust, open-source basis for advancing automated oncologic imaging, significantly reducing the need for large-scale manual annotations in clinical practice.

Related papers

• Mutual Distillation of Dual-Foundation Models for Semi-Supervised PET/CT Segmentation [8.784478648071547]
  Organ segmentation from PET/CT is critical for quantitative analysis and radiotherapy planning in oncology.<n>Semi-supervised learning (SSL) provides a practical and effective solution for developing deep models with limited labeled data.<n>We propose a mutual distillation framework that seamlessly exploits both structural and functional foundation models.
  arXiv  Detail & Related papers  (2026-06-14T05:34:26Z)
• The autoPET3 Challenge: Automated Lesion Segmentation in Whole-Body PET/CT $\unicode{x2013}$ Multitracer Multicenter Generalization [67.92281859568168]
  We report the design and results of the third autoPET challenge (MICCAI 2024)<n>It benchmarked automated lesion segmentation in whole-body PET/CT under a compositional generalization setting.<n>The top-ranked algorithm achieved a mean DSC of 0.66, FNV of 3.18 mL, and FPV of 2.78 mL across all four test conditions.
  arXiv  Detail & Related papers  (2026-05-07T07:07:09Z)
• A Generalist Foundation Model for Total-body PET/CT Enables Diagnostic Reporting and System-wide Metabolic Profiling [17.709593152803695]
  SDF-HOLO is a multimodal foundation model for holistic total-body PET/CT, pre-trained on more than 10,000 patients.<n>It decouples CT and PET representation learning with dual-stream encoders and couples them through a cross-modal interaction module.<n>The model enables system-wide metabolic profiling and reveals tumor-associated fingerprints of inter-organ metabolic network interactions.
  arXiv  Detail & Related papers  (2026-01-19T08:30:09Z)
• Unveiling and Bridging the Functional Perception Gap in MLLMs: Atomic Visual Alignment and Hierarchical Evaluation via PET-Bench [48.60251555171943]
  Multimodal Large Language Models (MLLMs) have demonstrated remarkable proficiency in tasks such as abnormality detection and report generation for anatomical modalities.<n>In this work, we quantify a fundamental functional perception gap: the inability of current vision encoders to decode functional tracer biodistribution independent of morphological priors.<n>We introduce PET-Bench, the first large-scale functional imaging benchmark comprising 52,308 hierarchical QA pairs from 9,732 multi-site, multi-tracer PET studies.<n>Our results demonstrate that AVA effectively bridges the perception gap, transforming CoT from a source of hallucination into a robust inference tool and improving diagnostic
  arXiv  Detail & Related papers  (2026-01-06T05:58:50Z)
• impuTMAE: Multi-modal Transformer with Masked Pre-training for Missing Modalities Imputation in Cancer Survival Prediction [75.43342771863837]
  We introduce impuTMAE, a novel transformer-based end-to-end approach with an efficient multimodal pre-training strategy.<n>It learns inter- and intra-modal interactions while simultaneously imputing missing modalities by reconstructing masked patches.<n>Our model is pre-trained on heterogeneous, incomplete data and fine-tuned for glioma survival prediction using TCGA-GBM/LGG and BraTS datasets.
  arXiv  Detail & Related papers  (2025-08-08T10:01:16Z)
• PathSegDiff: Pathology Segmentation using Diffusion model representations [63.20694440934692]
  We propose PathSegDiff, a novel approach for histopathology image segmentation that leverages Latent Diffusion Models (LDMs) as pre-trained featured extractors.<n>Our method utilizes a pathology-specific LDM, guided by a self-supervised encoder, to extract rich semantic information from H&E stained histopathology images.<n>Our experiments demonstrate significant improvements over traditional methods on the BCSS and GlaS datasets.
  arXiv  Detail & Related papers  (2025-04-09T14:58:21Z)
• Developing a PET/CT Foundation Model for Cross-Modal Anatomical and Functional Imaging [39.59895695500171]
  We introduce the Cross-Fraternal Twin Masked Autoencoder (FratMAE), a novel framework that effectively integrates whole-body anatomical and functional information.<n>FratMAE captures intricate cross-modal relationships and global uptake patterns, achieving superior performance on downstream tasks.
  arXiv  Detail & Related papers  (2025-03-04T17:49:07Z)
• From FDG to PSMA: A Hitchhiker's Guide to Multitracer, Multicenter Lesion Segmentation in PET/CT Imaging [0.9384264274298444]
  We present our solution for the autoPET III challenge, targeting multitracer, multicenter generalization using the nnU-Net framework with the ResEncL architecture. Key techniques include misalignment data augmentation and multi-modal pretraining across CT, MR, and PET datasets. Compared to the default nnU-Net, which achieved a Dice score of 57.61, our model significantly improved performance with a Dice score of 68.40, alongside a reduction in false positive (FPvol: 7.82) and false negative (FNvol: 10.35) volumes.
  arXiv  Detail & Related papers  (2024-09-14T16:39:17Z)
• Contrastive Diffusion Model with Auxiliary Guidance for Coarse-to-Fine PET Reconstruction [62.29541106695824]
  This paper presents a coarse-to-fine PET reconstruction framework that consists of a coarse prediction module (CPM) and an iterative refinement module (IRM) By delegating most of the computational overhead to the CPM, the overall sampling speed of our method can be significantly improved. Two additional strategies, i.e., an auxiliary guidance strategy and a contrastive diffusion strategy, are proposed and integrated into the reconstruction process.
  arXiv  Detail & Related papers  (2023-08-20T04:10:36Z)
• Many-to-One Distribution Learning and K-Nearest Neighbor Smoothing for Thoracic Disease Identification [83.6017225363714]
  deep learning has become the most powerful computer-aided diagnosis technology for improving disease identification performance. For chest X-ray imaging, annotating large-scale data requires professional domain knowledge and is time-consuming. In this paper, we propose many-to-one distribution learning (MODL) and K-nearest neighbor smoothing (KNNS) methods to improve a single model's disease identification performance.
  arXiv  Detail & Related papers  (2021-02-26T02:29:30Z)
• Multimodal Spatial Attention Module for Targeting Multimodal PET-CT Lung Tumor Segmentation [11.622615048002567]
  Multimodal spatial attention module (MSAM) learns to emphasize regions related to tumors. MSAM can be applied to common backbone architectures and trained end-to-end.
  arXiv  Detail & Related papers  (2020-07-29T10:27:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.