pFedSAM: Personalized Federated Learning of Segment Anything Model for Medical Image Segmentation

• URL: http://arxiv.org/abs/2509.15638v1
• Date: Fri, 19 Sep 2025 06:05:35 GMT
• Title: pFedSAM: Personalized Federated Learning of Segment Anything Model for Medical Image Segmentation
• Authors: Tong Wang, Xingyue Zhao, Linghao Zhuang, Haoyu Zhao, Jiayi Yin, Yuyang He, Gang Yu, Bo Lin,
• Abstract summary: We present the first personalized federated SAM framework tailored for heterogeneous data scenarios in medical image segmentation.<n>Our framework integrates two key innovations: (1) a personalized strategy that aggregates only the global parameters to capture cross-client commonalities while retaining the designed L-MoE component to preserve domain-specific features; and (2) a decoupled global-local fine-tuning mechanism that leverages a teacher-student paradigm via knowledge distillation to bridge the gap between the global shared model and the personalized local models, thereby mitigating overgeneralization.
• Score: 23.86147005761286
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Medical image segmentation is crucial for computer-aided diagnosis, yet privacy constraints hinder data sharing across institutions. Federated learning addresses this limitation, but existing approaches often rely on lightweight architectures that struggle with complex, heterogeneous data. Recently, the Segment Anything Model (SAM) has shown outstanding segmentation capabilities; however, its massive encoder poses significant challenges in federated settings. In this work, we present the first personalized federated SAM framework tailored for heterogeneous data scenarios in medical image segmentation. Our framework integrates two key innovations: (1) a personalized strategy that aggregates only the global parameters to capture cross-client commonalities while retaining the designed L-MoE (Localized Mixture-of-Experts) component to preserve domain-specific features; and (2) a decoupled global-local fine-tuning mechanism that leverages a teacher-student paradigm via knowledge distillation to bridge the gap between the global shared model and the personalized local models, thereby mitigating overgeneralization. Extensive experiments on two public datasets validate that our approach significantly improves segmentation performance, achieves robust cross-domain adaptation, and reduces communication overhead.

Related papers

• Federated Learning for Cross-Modality Medical Image Segmentation via Augmentation-Driven Generalization [0.0]
  In this work, we consider a realistic FL scenario where each client holds single-modality data (CT or MRI)<n>We evaluate convolution-based spatial augmentation, frequency-domain manipulation, domain-specific normalization, and global intensity nonlinear (GIN) augmentation.<n>Our federated approach achieves 93-98% of centralized training accuracy, demonstrating strong cross-modality generalization without compromising data privacy.
  arXiv  Detail & Related papers  (2026-02-24T11:13:01Z)
• Towards Federated Clustering: A Client-wise Private Graph Aggregation Framework [57.04850867402913]
  Federated clustering addresses the challenge of extracting patterns from decentralized, unlabeled data.<n>We propose Structural Privacy-Preserving Federated Graph Clustering (SPP-FGC), a novel algorithm that innovatively leverages local structural graphs as the primary medium for privacy-preserving knowledge sharing.<n>Our framework achieves state-of-the-art performance, improving clustering accuracy by up to 10% (NMI) over federated baselines while maintaining provable privacy guarantees.
  arXiv  Detail & Related papers  (2025-11-14T03:05:22Z)
• Adversarial Versus Federated: An Adversarial Learning based Multi-Modality Cross-Domain Federated Medical Segmentation [30.99222543580891]
  Federated learning enables collaborative training of machine learning models among different clients.<n>We propose a new Federated Domain Adaptation (FedDA) segmentation training framework.<n>Our proposed FedDA substantially achieves cross-domain federated aggregation, endowing single modality client with cross-modality processing capabilities.
  arXiv  Detail & Related papers  (2025-09-28T14:26:04Z)
• Segment Concealed Objects with Incomplete Supervision [63.637733655439334]
  Incompletely-Supervised Concealed Object (ISCOS) involves segmenting objects that seamlessly blend into their surrounding environments.<n>This task remains highly challenging due to the limited supervision provided by the incompletely annotated training data.<n>In this paper, we introduce the first unified method for ISCOS to address these challenges.
  arXiv  Detail & Related papers  (2025-06-10T16:25:15Z)
• Federated Client-tailored Adapter for Medical Image Segmentation [21.964553228831427]
  We propose a novel Federated Client-tailored Adapter (FCA) framework for medical image segmentation.<n>FCA achieves stable and client-tailored adaptive segmentation without sharing sensitive local data.<n>We develop two client-tailored federated updating strategies that adaptively decompose the adapter into common and individual components.
  arXiv  Detail & Related papers  (2025-04-25T02:20:25Z)
• FedSemiDG: Domain Generalized Federated Semi-supervised Medical Image Segmentation [19.87797382888023]
  Medical image segmentation is challenging due to the diversity of medical images and the lack of labeled data.<n>We present a novel framework, Federated Generalization-Aware SemiSupervised Learning (FGASL), to address the challenges in FedSemiDG.<n>Our method significantly outperforms state-of-the-art FSSL and domain generalization approaches, achieving robust generalization on unseen domains.
  arXiv  Detail & Related papers  (2025-01-13T14:54:49Z)
• Unifying and Personalizing Weakly-supervised Federated Medical Image Segmentation via Adaptive Representation and Aggregation [1.121358474059223]
  Federated learning (FL) enables multiple sites to collaboratively train powerful deep models without compromising data privacy and security. Weakly supervised segmentation, which uses sparsely-grained supervision, is increasingly being paid attention to due to its great potential of reducing annotation costs. We propose a novel personalized FL framework for medical image segmentation, named FedICRA, which uniformly leverages heterogeneous weak supervision.
  arXiv  Detail & Related papers  (2023-04-12T06:32:08Z)
• DRFLM: Distributionally Robust Federated Learning with Inter-client Noise via Local Mixup [58.894901088797376]
  federated learning has emerged as a promising approach for training a global model using data from multiple organizations without leaking their raw data. We propose a general framework to solve the above two challenges simultaneously. We provide comprehensive theoretical analysis including robustness analysis, convergence analysis, and generalization ability.
  arXiv  Detail & Related papers  (2022-04-16T08:08:29Z)
• Adapt to Adaptation: Learning Personalization for Cross-Silo Federated Learning [6.0088002781256185]
  Conventional federated learning aims to train a global model for a federation of clients with decentralized data. The distribution shift across non-IID datasets, also known as the data heterogeneity, often poses a challenge for this one-global-model-fits-all solution. We propose APPLE, a personalized cross-silo FL framework that adaptively learns how much each client can benefit from other clients' models.
  arXiv  Detail & Related papers  (2021-10-15T22:23:14Z)
• Source-Free Open Compound Domain Adaptation in Semantic Segmentation [99.82890571842603]
  In SF-OCDA, only the source pre-trained model and the target data are available to learn the target model. We propose the Cross-Patch Style Swap (CPSS) to diversify samples with various patch styles in the feature-level. Our method produces state-of-the-art results on the C-Driving dataset.
  arXiv  Detail & Related papers  (2021-06-07T08:38:41Z)
• FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Space [63.43592895652803]
  Federated learning allows distributed medical institutions to collaboratively learn a shared prediction model with privacy protection. While at clinical deployment, the models trained in federated learning can still suffer from performance drop when applied to completely unseen hospitals outside the federation. We present a novel approach, named as Episodic Learning in Continuous Frequency Space (ELCFS), for this problem. The effectiveness of our method is demonstrated with superior performance over state-of-the-arts and in-depth ablation experiments on two medical image segmentation tasks.
  arXiv  Detail & Related papers  (2021-03-10T13:05:23Z)
• Federated Unsupervised Representation Learning [56.715917111878106]
  We formulate a new problem in federated learning called Federated Unsupervised Representation Learning (FURL) to learn a common representation model without supervision. FedCA is composed of two key modules: dictionary module to aggregate the representations of samples from each client and share with all clients for consistency of representation space and alignment module to align the representation of each client on a base model trained on a public data.
  arXiv  Detail & Related papers  (2020-10-18T13:28:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.