HASA: Subnet Allocation for Compute-Constrained Model-Heterogeneous Federated Learning

• URL: http://arxiv.org/abs/2606.07621v1
• Date: Sat, 30 May 2026 11:21:48 GMT
• Title: HASA: Subnet Allocation for Compute-Constrained Model-Heterogeneous Federated Learning
• Abstract summary: Edge services increasingly use federated learning to personalize on-device models while keeping sensitive data local.<n>In practice, deployments must handle in both client resources and local data distributions.<n>This paper proposes Heterogeneity-Aware Subnet Allocation (HASA), a train-only rule that assigns widths based on client heterogeneity scores.
• Score: 6.169964580735435
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Edge services increasingly use federated learning to personalize on-device models while keeping sensitive data local. In practice, deployments must handle heterogeneity in both client resources and local data distributions. Model-heterogeneous federated learning lowers client cost by allowing each client to train a subnet of a shared supernet, but most subnet-allocation policies are driven by device constraints and do not explicitly account for statistical heterogeneity. This paper proposes Heterogeneity-Aware Subnet Allocation (HASA), a train-only rule that assigns subnet widths based on client heterogeneity scores computed from local training data while enforcing a fixed size-weighted compute budget. This design enables budget-matched comparisons with alternative allocation policies. On an article-title next-word prediction benchmark with seven clients, HASA improves unweighted mean client test accuracy over uniform allocation across 10 matched seeds, increasing mean client test accuracy from 13.82 percent to 14.32 percent, and improves worst-client accuracy on average. In a matched-budget comparison with representative partial-training baselines, HASA achieves the strongest worst-client and tail-client accuracy on this benchmark. A directionality ablation shows that assigning smaller subnets to more heterogeneous clients degrades both mean and tail performance. A cross-domain image-classification study further shows that the effectiveness of heterogeneity-aware allocation depends on how well the heterogeneity score reflects clients' need for additional model width.

Related papers

• When Is Shallow Enough? Adaptive Split Federated Learning with Client-Specific Sufficiency Estimation [8.3131553967982]
  Under client heterogeneity, the conventional static split strategy may be suboptimal.<n>We propose textscFedSGA, a textbfSufficiency-textbfGuided textbfAdaptive split textbfFederated learning framework.
  arXiv  Detail & Related papers  (2026-08-16T09:06:28Z)
• Beyond Parameter Space: NTK-Guided Personalized Aggregation for Robust Federated Learning [1.3924112497839525]
  Federated learning enables collaborative model training across distributed clients while keeping data local.<n>A central challenge is determining which client updates are beneficial for aggregation with respect to each client's target domain.<n>We propose a federated learning framework that performs update selection in function space.
  arXiv  Detail & Related papers  (2026-08-12T14:29:43Z)
• Replacing Parameters with Preferences: Federated Alignment of Heterogeneous Vision-Language Models [63.70401095689976]
  Vision-Language Models (VLMs) have broad potential in privacy-sensitive domains such as healthcare and finance.<n>We propose MoR, a federated alignment framework that combines GRPO with Mixture-of-Rewards for heterogeneousVLMs.<n>MoR consistently outperforms federated alignment baselines in generalization and cross-client adaptability.
  arXiv  Detail & Related papers  (2026-05-05T07:02:50Z)
• Replacing Parameters with Preferences: Federated Alignment of Heterogeneous Vision-Language Models [63.70401095689976]
  We argue that replacing parameters with preferences represents a more scalable and privacy-preserving future.<n>We propose MoR, a federated alignment framework based on GRPO with Mixture-of-Rewards for heterogeneous VLMs.<n>MoR consistently outperforms federated alignment baselines in generalization, robustness, and cross-client adaptability.
  arXiv  Detail & Related papers  (2026-01-31T03:11:51Z)
• Latte: Collaborative Test-Time Adaptation of Vision-Language Models in Federated Learning [66.35456997311742]
  Test-time adaptation with pre-trained vision-language models has gained increasing attention for addressing distribution shifts during testing.<n>Existing test-time adaptation methods are typically designed for a single domain with abundant data.<n>We propose Latte, a novel framework where each client maintains a local memory to store embeddings from its own historical test data.
  arXiv  Detail & Related papers  (2025-07-29T04:27:29Z)
• Modality Alignment Meets Federated Broadcasting [9.752555511824593]
  Federated learning (FL) has emerged as a powerful approach to safeguard data privacy by training models across distributed edge devices without centralizing local data. This paper introduces a novel FL framework leveraging modality alignment, where a text encoder resides on the server, and image encoders operate on local devices.
  arXiv  Detail & Related papers  (2024-11-24T13:30:03Z)
• Towards Instance-adaptive Inference for Federated Learning [80.38701896056828]
  Federated learning (FL) is a distributed learning paradigm that enables multiple clients to learn a powerful global model by aggregating local training. In this paper, we present a novel FL algorithm, i.e., FedIns, to handle intra-client data heterogeneity by enabling instance-adaptive inference in the FL framework. Our experiments show that our FedIns outperforms state-of-the-art FL algorithms, e.g., a 6.64% improvement against the top-performing method with less than 15% communication cost on Tiny-ImageNet.
  arXiv  Detail & Related papers  (2023-08-11T09:58:47Z)
• Boosting Federated Learning Convergence with Prototype Regularization [46.23872011375391]
  This paper introduces a prototype-based regularization strategy to address the heterogeneous data distribution among clients. The experimental results on MNIST and Fashion-MNIST show that our proposal achieves improvements of 3.3% and 8.9% in average test accuracy.
  arXiv  Detail & Related papers  (2023-07-20T04:35:50Z)
• Efficient Distribution Similarity Identification in Clustered Federated Learning via Principal Angles Between Client Data Subspaces [59.33965805898736]
  Clustered learning has been shown to produce promising results by grouping clients into clusters. Existing FL algorithms are essentially trying to group clients together with similar distributions. Prior FL algorithms attempt similarities indirectly during training.
  arXiv  Detail & Related papers  (2022-09-21T17:37:54Z)
• Federated Multi-Target Domain Adaptation [99.93375364579484]
  Federated learning methods enable us to train machine learning models on distributed user data while preserving its privacy. We consider a more practical scenario where the distributed client data is unlabeled, and a centralized labeled dataset is available on the server. We propose an effective DualAdapt method to address the new challenges.
  arXiv  Detail & Related papers  (2021-08-17T17:53:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.