Hierarchical Knowledge Structuring for Effective Federated Learning in Heterogeneous Environments

• URL: http://arxiv.org/abs/2504.03505v1
• Date: Fri, 04 Apr 2025 15:06:02 GMT
• Title: Hierarchical Knowledge Structuring for Effective Federated Learning in Heterogeneous Environments
• Authors: Wai Fong Tam, Qilei Li, Ahmed M. Abdelmonie,
• Abstract summary: Federated learning enables collaborative model training across distributed entities while maintaining individual data privacy.<n>Recent efforts leverage logit-based knowledge aggregation and distillation to overcome these issues.<n>We propose a Hierarchical Knowledge Structuring (HKS) framework that formulates sample logits into a multi-granularity codebook.
• Score: 0.6144680854063939
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Federated learning enables collaborative model training across distributed entities while maintaining individual data privacy. A key challenge in federated learning is balancing the personalization of models for local clients with generalization for the global model. Recent efforts leverage logit-based knowledge aggregation and distillation to overcome these issues. However, due to the non-IID nature of data across diverse clients and the imbalance in the client's data distribution, directly aggregating the logits often produces biased knowledge that fails to apply to individual clients and obstructs the convergence of local training. To solve this issue, we propose a Hierarchical Knowledge Structuring (HKS) framework that formulates sample logits into a multi-granularity codebook to represent logits from personalized per-sample insights to globalized per-class knowledge. The unsupervised bottom-up clustering method is leveraged to enable the global server to provide multi-granularity responses to local clients. These responses allow local training to integrate supervised learning objectives with global generalization constraints, which results in more robust representations and improved knowledge sharing in subsequent training rounds. The proposed framework's effectiveness is validated across various benchmarks and model architectures.

Related papers

• Enhancing Federated Learning Through Secure Cluster-Weighted Client Aggregation [4.869042695112397]
  Federated learning (FL) has emerged as a promising paradigm in machine learning. In FL, a global model is trained iteratively on local datasets residing on individual devices. This paper introduces a novel FL framework, ClusterGuardFL, that employs dissimilarity scores, k-means clustering, and reconciliation confidence scores to dynamically assign weights to client updates.
  arXiv  Detail & Related papers  (2025-03-29T04:29:24Z)
• FedRIR: Rethinking Information Representation in Federated Learning [7.765917932796046]
  Mobile and Web-of-Things (WoT) devices at the network edge generate vast amounts of data for machine learning applications.<n>Mobile and Web-of-Things (WoT) devices at the network edge generate vast amounts of data for machine learning applications.
  arXiv  Detail & Related papers  (2025-02-02T17:17:29Z)
• Adversarial Federated Consensus Learning for Surface Defect Classification Under Data Heterogeneity in IIoT [8.48069043458347]
  It's difficult to collect and centralize sufficient training data from various entities in Industrial Internet of Things (IIoT) Federated learning (FL) provides a solution by enabling collaborative global model training across clients. We propose a novel personalized FL approach, named Adversarial Federated Consensus Learning (AFedCL)
  arXiv  Detail & Related papers  (2024-09-24T03:59:32Z)
• Unlocking the Potential of Prompt-Tuning in Bridging Generalized and Personalized Federated Learning [49.72857433721424]
  Vision Transformers (ViT) and Visual Prompt Tuning (VPT) achieve state-of-the-art performance with improved efficiency in various computer vision tasks. We present a novel algorithm, SGPT, that integrates Generalized FL (GFL) and Personalized FL (PFL) approaches by employing a unique combination of both shared and group-specific prompts.
  arXiv  Detail & Related papers  (2023-10-27T17:22:09Z)
• Rethinking Client Drift in Federated Learning: A Logit Perspective [125.35844582366441]
  Federated Learning (FL) enables multiple clients to collaboratively learn in a distributed way, allowing for privacy protection. We find that the difference in logits between the local and global models increases as the model is continuously updated. We propose a new algorithm, named FedCSD, a Class prototype Similarity Distillation in a federated framework to align the local and global models.
  arXiv  Detail & Related papers  (2023-08-20T04:41:01Z)
• Personalized Federated Learning via Amortized Bayesian Meta-Learning [21.126405589760367]
  We introduce a new perspective on personalized federated learning through Amortized Bayesian Meta-Learning. Specifically, we propose a novel algorithm called emphFedABML, which employs hierarchical variational inference across clients. Our theoretical analysis provides an upper bound on the average generalization error and guarantees the generalization performance on unseen data.
  arXiv  Detail & Related papers  (2023-07-05T11:58:58Z)
• Knowledge-Aware Federated Active Learning with Non-IID Data [75.98707107158175]
  We propose a federated active learning paradigm to efficiently learn a global model with limited annotation budget. The main challenge faced by federated active learning is the mismatch between the active sampling goal of the global model on the server and that of the local clients. We propose Knowledge-Aware Federated Active Learning (KAFAL), which consists of Knowledge-Specialized Active Sampling (KSAS) and Knowledge-Compensatory Federated Update (KCFU)
  arXiv  Detail & Related papers  (2022-11-24T13:08:43Z)
• Straggler-Resilient Personalized Federated Learning [55.54344312542944]
  Federated learning allows training models from samples distributed across a large network of clients while respecting privacy and communication restrictions. We develop a novel algorithmic procedure with theoretical speedup guarantees that simultaneously handles two of these hurdles. Our method relies on ideas from representation learning theory to find a global common representation using all clients' data and learn a user-specific set of parameters leading to a personalized solution for each client.
  arXiv  Detail & Related papers  (2022-06-05T01:14:46Z)
• Personalized Federated Learning through Local Memorization [10.925242558525683]
  Federated learning allows clients to collaboratively learn statistical models while keeping their data local. Recent personalized federated learning methods train a separate model for each client while still leveraging the knowledge available at other clients. We show on a suite of federated datasets that this approach achieves significantly higher accuracy and fairness than state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-11-17T19:40:07Z)
• Decentralised Person Re-Identification with Selective Knowledge Aggregation [56.40855978874077]
  Existing person re-identification (Re-ID) methods mostly follow a centralised learning paradigm which shares all training data to a collection for model learning. Two recent works have introduced decentralised (federated) Re-ID learning for constructing a globally generalised model (server) However, these methods are poor on how to adapt the generalised model to maximise its performance on individual client domain Re-ID tasks. We present a new Selective Knowledge Aggregation approach to decentralised person Re-ID to optimise the trade-off between model personalisation and generalisation.
  arXiv  Detail & Related papers  (2021-10-21T18:09:53Z)
• GRP-FED: Addressing Client Imbalance in Federated Learning via Global-Regularized Personalization [6.592268037926868]
  We present Global-Regularized Personalization (GRP-FED) to tackle the data imbalanced issue. With adaptive aggregation, the global model treats multiple clients fairly and mitigates the global long-tailed issue. Our results show that our GRP-FED improves under both global and local scenarios.
  arXiv  Detail & Related papers  (2021-08-31T14:09:04Z)
• Exploiting Shared Representations for Personalized Federated Learning [54.65133770989836]
  We propose a novel federated learning framework and algorithm for learning a shared data representation across clients and unique local heads for each client. Our algorithm harnesses the distributed computational power across clients to perform many local-updates with respect to the low-dimensional local parameters for every update of the representation. This result is of interest beyond federated learning to a broad class of problems in which we aim to learn a shared low-dimensional representation among data distributions.
  arXiv  Detail & Related papers  (2021-02-14T05:36:25Z)
• Decentralised Learning from Independent Multi-Domain Labels for Person Re-Identification [69.29602103582782]
  Deep learning has been successful for many computer vision tasks due to the availability of shared and centralised large-scale training data. However, increasing awareness of privacy concerns poses new challenges to deep learning, especially for person re-identification (Re-ID) We propose a novel paradigm called Federated Person Re-Identification (FedReID) to construct a generalisable global model (a central server) by simultaneously learning with multiple privacy-preserved local models (local clients) This client-server collaborative learning process is iteratively performed under privacy control, enabling FedReID to realise decentralised learning without sharing distributed data nor collecting any
  arXiv  Detail & Related papers  (2020-06-07T13:32:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.