Knowledge Augmentation in Federation: Rethinking What Collaborative Learning Can Bring Back to Decentralized Data

• URL: http://arxiv.org/abs/2503.03140v2
• Date: Fri, 07 Mar 2025 02:57:44 GMT
• Title: Knowledge Augmentation in Federation: Rethinking What Collaborative Learning Can Bring Back to Decentralized Data
• Authors: Wentai Wu, Ligang He, Saiqin Long, Ahmed M. Abdelmoniem, Yingliang Wu, Rui Mao,
• Abstract summary: We present a knowledge-centric paradigm termed Knowledge Augmentation in Federation (KAF)<n>We provide the suggested system architecture, formulate the prototypical optimization objective, and review emerging studies that employ methodologies suitable for KAF.<n>We highlight several challenges and open questions that deserve more attention from the community.
• Score: 11.521866946305986
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Data, as an observable form of knowledge, has become one of the most important factors of production for the development of Artificial Intelligence (AI). Meanwhile, increasing legislation and regulations on private and proprietary information results in scattered data sources also known as the "data islands". Although some collaborative learning paradigms such as Federated Learning (FL) can enable privacy-preserving training over decentralized data, they have inherent deficiencies in fairness, costs and reproducibility because of being learning-centric, which greatly limits the way how participants cooperate with each other. In light of this, we present a knowledge-centric paradigm termed Knowledge Augmentation in Federation (KAF), with focus on how to enhance local knowledge through collaborative effort. We provide the suggested system architecture, formulate the prototypical optimization objective, and review emerging studies that employ methodologies suitable for KAF. On our roadmap, with a three-way categorization we describe the methods for knowledge expansion, knowledge filtering, and label and feature space correction in the federation. Further, we highlight several challenges and open questions that deserve more attention from the community. With our investigation, we intend to offer new insights for what collaborative learning can bring back to decentralized data.

Related papers

• LEKA:LLM-Enhanced Knowledge Augmentation [24.552995956148145]
  Humans excel in analogical learning and knowledge transfer. Models would transition from passively acquiring to actively accessing and learning from knowledge. We develop a knowledge augmentation method LEKA for knowledge transfer.
  arXiv  Detail & Related papers  (2025-01-29T17:44:57Z)
• Promise and Peril of Collaborative Code Generation Models: Balancing Effectiveness and Memorization [13.949319911378826]
  This study investigates key factors that impact the effectiveness of collaborative training methods in code next-token prediction. We show that the size and diversity of code datasets are pivotal factors influencing the success of collaboratively trained code models. Our findings highlight the persistent risk of data leakage during inference, even when training data remains unseen.
  arXiv  Detail & Related papers  (2024-09-18T14:30:48Z)
• KnFu: Effective Knowledge Fusion [5.305607095162403]
  Federated Learning (FL) has emerged as a prominent alternative to the traditional centralized learning approach. The paper proposes Effective Knowledge Fusion (KnFu) algorithm that evaluates knowledge of local models to only fuse semantic neighbors' effective knowledge for each client. A key conclusion of the work is that in scenarios with large and highly heterogeneous local datasets, local training could be preferable to knowledge fusion-based solutions.
  arXiv  Detail & Related papers  (2024-03-18T15:49:48Z)
• Private Knowledge Sharing in Distributed Learning: A Survey [50.51431815732716]
  The rise of Artificial Intelligence has revolutionized numerous industries and transformed the way society operates. It is crucial to utilize information in learning processes that are either distributed or owned by different entities. Modern data-driven services have been developed to integrate distributed knowledge entities into their outcomes.
  arXiv  Detail & Related papers  (2024-02-08T07:18:23Z)
• Personalized Federated Learning with Attention-based Client Selection [57.71009302168411]
  We propose FedACS, a new PFL algorithm with an Attention-based Client Selection mechanism. FedACS integrates an attention mechanism to enhance collaboration among clients with similar data distributions. Experiments on CIFAR10 and FMNIST validate FedACS's superiority.
  arXiv  Detail & Related papers  (2023-12-23T03:31:46Z)
• Exploring Federated Unlearning: Analysis, Comparison, and Insights [101.64910079905566]
  federated unlearning enables the selective removal of data from models trained in federated systems.<n>This paper examines existing federated unlearning approaches, examining their algorithmic efficiency, impact on model accuracy, and effectiveness in preserving privacy.<n>We propose the OpenFederatedUnlearning framework, a unified benchmark for evaluating federated unlearning methods.
  arXiv  Detail & Related papers  (2023-10-30T01:34:33Z)
• When Decentralized Optimization Meets Federated Learning [41.58479981773202]
  Federated learning is a new learning paradigm for extracting knowledge from distributed data. Most existing federated learning approaches concentrate on the centralized setting, which is vulnerable to a single-point failure. An alternative strategy for addressing this issue is the decentralized communication topology.
  arXiv  Detail & Related papers  (2023-06-05T03:51:14Z)
• Selective Knowledge Sharing for Privacy-Preserving Federated Distillation without A Good Teacher [52.2926020848095]
  Federated learning is vulnerable to white-box attacks and struggles to adapt to heterogeneous clients. This paper proposes a selective knowledge sharing mechanism for FD, termed Selective-FD.
  arXiv  Detail & Related papers  (2023-04-04T12:04:19Z)
• Combating Exacerbated Heterogeneity for Robust Models in Federated Learning [91.88122934924435]
  Combination of adversarial training and federated learning can lead to the undesired robustness deterioration. We propose a novel framework called Slack Federated Adversarial Training (SFAT) We verify the rationality and effectiveness of SFAT on various benchmarked and real-world datasets.
  arXiv  Detail & Related papers  (2023-03-01T06:16:15Z)
• Heterogeneous Federated Knowledge Graph Embedding Learning and Unlearning [14.063276595895049]
  Federated Learning (FL) is a paradigm to train a global machine learning model across distributed clients without sharing raw data. We propose FedLU, a novel FL framework for heterogeneous KG embedding learning and unlearning. We show that FedLU achieves superior results in both link prediction and knowledge forgetting.
  arXiv  Detail & Related papers  (2023-02-04T02:44:48Z)
• Exploring Semantic Attributes from A Foundation Model for Federated Learning of Disjoint Label Spaces [46.59992662412557]
  In this work, we consider transferring mid-level semantic knowledge (such as attribute) which is not sensitive to specific objects of interest. We formulate a new Federated Zero-Shot Learning (FZSL) paradigm to learn mid-level semantic knowledge at multiple local clients. To improve model discriminative ability, we propose to explore semantic knowledge augmentation from external knowledge.
  arXiv  Detail & Related papers  (2022-08-29T10:05:49Z)
• Knowledge-Injected Federated Learning [44.89926234630289]
  Federated learning is an emerging technique for training models from decentralized data sets. In many applications, data owners participating in the federated learning system hold not only the data but also a set of domain knowledge. We propose a federated learning framework that allows the injection of participants' domain knowledge.
  arXiv  Detail & Related papers  (2022-08-16T04:23:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.