Federated Ensemble Learning with Progressive Model Personalization

• URL: http://arxiv.org/abs/2602.14244v1
• Date: Sun, 15 Feb 2026 17:35:52 GMT
• Title: Federated Ensemble Learning with Progressive Model Personalization
• Authors: Ala Emrani, Amir Najafi, Abolfazl Motahari,
• Abstract summary: Federated Learning provides a privacy-preserving paradigm for distributed learning, but suffers from statistical heterogeneity across clients.<n>A widely adopted approach in PFL decomposes neural networks into a shared feature extractor and client-specific heads.<n>Most existing methods rely on rigid, shallow heads, and therefore fail to navigate this tradeoff in a principled manner.<n>We propose a boosting-inspired framework that enables a smooth control of this tradeoff.
• Score: 1.1045253485823572
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Federated Learning provides a privacy-preserving paradigm for distributed learning, but suffers from statistical heterogeneity across clients. Personalized Federated Learning (PFL) mitigates this issue by considering client-specific models. A widely adopted approach in PFL decomposes neural networks into a shared feature extractor and client-specific heads. While effective, this design induces a fundamental tradeoff: deep or expressive shared components hinder personalization, whereas large local heads exacerbate overfitting under limited per-client data. Most existing methods rely on rigid, shallow heads, and therefore fail to navigate this tradeoff in a principled manner. In this work, we propose a boosting-inspired framework that enables a smooth control of this tradeoff. Instead of training a single personalized model, we construct an ensemble of $T$ models for each client. Across boosting iterations, the depth of the personalized component are progressively increased, while its effective complexity is systematically controlled via low-rank factorization or width shrinkage. This design simultaneously limits overfitting and substantially reduces per-client bias by allowing increasingly expressive personalization. We provide theoretical analysis that establishes generalization bounds with favorable dependence on the average local sample size and the total number of clients. Specifically, we prove that the complexity of the shared layers is effectively suppressed, while the dependence on the boosting horizon $T$ is controlled through parameter reduction. Notably, we provide a novel nonlinear generalization guarantee for decoupled PFL models. Extensive experiments on benchmark and real-world datasets (e.g., EMNIST, CIFAR-10/100, and Sent140) demonstrate that the proposed framework consistently outperforms state-of-the-art PFL methods under heterogeneous data distributions.

Related papers

• CO-PFL: Contribution-Oriented Personalized Federated Learning for Heterogeneous Networks [51.43780477302533]
  Contribution-Oriented PFL (CO-PFL) is a novel algorithm that dynamically estimates each client's contribution for global aggregation.<n>CO-PFL consistently surpasses state-of-the-art methods in robustness in personalization accuracy, robustness, scalability and convergence stability.
  arXiv  Detail & Related papers  (2025-10-23T05:10:06Z)
• FedPPA: Progressive Parameter Alignment for Personalized Federated Learning [0.9931624906346306]
  Federated Learning (FL) is designed as a decentralized, privacy-preserving machine learning paradigm that enables multiple clients to collaboratively train a model without sharing their data.<n>In real-world scenarios, clients often have heterogeneous computational resources and hold non-independent and identically distributed data (non-IID), which poses significant challenges during training.<n>We propose Progressive Alignment (FedPPA), which progressively aligns the weights of common layers across clients with the global model's weights.<n>Experiments on three image classification datasets, including MNIST, FMNIST, and CIFAR-10, demonstrate that FedPPA consistently outperforms existing FL algorithms,
  arXiv  Detail & Related papers  (2025-10-16T14:03:05Z)
• Hypernetworks for Model-Heterogeneous Personalized Federated Learning [13.408669475480824]
  We propose a server-side hypernetwork that takes client-specific embedding vectors as input and outputs personalized parameters tailored to each client's heterogeneous model.<n>To promote knowledge sharing and reduce computation, we introduce a multi-head structure within the hypernetwork, allowing clients with similar model sizes to share heads.<n>Our framework does not rely on external datasets and does not require disclosure of client model architectures.
  arXiv  Detail & Related papers  (2025-07-30T02:24:26Z)
• pMixFed: Efficient Personalized Federated Learning through Adaptive Layer-Wise Mixup [12.610705758718183]
  pMixFed is a dynamic, layer-wise PFL approach that integrates $textitpMixFed$ between shared global and personalized local models.<n>Our method introduces an adaptive strategy for partitioning between personalized and shared layers, a gradual transition of personalization degree, improved generalization across clients, and a novel aggregation mechanism to mitigate catastrophic forgetting.
  arXiv  Detail & Related papers  (2025-01-19T10:15:36Z)
• Client-Centric Federated Adaptive Optimization [78.30827455292827]
  Federated Learning (FL) is a distributed learning paradigm where clients collaboratively train a model while keeping their own data private.<n>We propose Federated-Centric Adaptive Optimization, which is a class of novel federated optimization approaches.
  arXiv  Detail & Related papers  (2025-01-17T04:00:50Z)
• Hybrid-Regularized Magnitude Pruning for Robust Federated Learning under Covariate Shift [2.298932494750101]
  We show that inconsistencies in client-side training distributions substantially degrade the performance of federated learning models.<n>We propose a novel FL framework using a combination of pruning and regularisation of clients' training to improve the sparsity, redundancy, and robustness of neural connections.
  arXiv  Detail & Related papers  (2024-12-19T16:22:37Z)
• Decentralized Directed Collaboration for Personalized Federated Learning [39.29794569421094]
  We concentrate on the Decentralized Personalized Learning (DPFL) that performs distributed training model computation. We propose a directed collaboration framework by incorporating textbfDecentralized textbfFederated textbfPartial textbfGradient textbfPedGP.
  arXiv  Detail & Related papers  (2024-05-28T06:52:19Z)
• FedImpro: Measuring and Improving Client Update in Federated Learning [77.68805026788836]
  Federated Learning (FL) models often experience client drift caused by heterogeneous data. We present an alternative perspective on client drift and aim to mitigate it by generating improved local models.
  arXiv  Detail & Related papers  (2024-02-10T18:14:57Z)
• Towards Instance-adaptive Inference for Federated Learning [80.38701896056828]
  Federated learning (FL) is a distributed learning paradigm that enables multiple clients to learn a powerful global model by aggregating local training. In this paper, we present a novel FL algorithm, i.e., FedIns, to handle intra-client data heterogeneity by enabling instance-adaptive inference in the FL framework. Our experiments show that our FedIns outperforms state-of-the-art FL algorithms, e.g., a 6.64% improvement against the top-performing method with less than 15% communication cost on Tiny-ImageNet.
  arXiv  Detail & Related papers  (2023-08-11T09:58:47Z)
• Confidence-aware Personalized Federated Learning via Variational Expectation Maximization [34.354154518009956]
  We present a novel framework for personalized Federated Learning (PFL) PFL is a distributed learning scheme to train a shared model across clients. We present a novel framework for PFL based on hierarchical modeling and variational inference.
  arXiv  Detail & Related papers  (2023-05-21T20:12:27Z)
• Personalized Federated Learning under Mixture of Distributions [98.25444470990107]
  We propose a novel approach to Personalized Federated Learning (PFL), which utilizes Gaussian mixture models (GMM) to fit the input data distributions across diverse clients. FedGMM possesses an additional advantage of adapting to new clients with minimal overhead, and it also enables uncertainty quantification. Empirical evaluations on synthetic and benchmark datasets demonstrate the superior performance of our method in both PFL classification and novel sample detection.
  arXiv  Detail & Related papers  (2023-05-01T20:04:46Z)
• QuPeD: Quantized Personalization via Distillation with Applications to Federated Learning [8.420943739336067]
  federated learning (FL) aims to train a single global model while collaboratively using multiple clients and a server. We introduce a textitquantized and textitpersonalized FL algorithm QuPeD that facilitates collective (personalized model compression) training. Numerically, we validate that QuPeD outperforms competing personalized FL methods, FedAvg, and local training of clients in various heterogeneous settings.
  arXiv  Detail & Related papers  (2021-07-29T10:55:45Z)
• Toward Understanding the Influence of Individual Clients in Federated Learning [52.07734799278535]
  Federated learning allows clients to jointly train a global model without sending their private data to a central server. We defined a new notion called em-Influence, quantify this influence over parameters, and proposed an effective efficient model to estimate this metric.
  arXiv  Detail & Related papers  (2020-12-20T14:34:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.