Knowledge-Injected Federated Learning

• URL: http://arxiv.org/abs/2208.07530v1
• Date: Tue, 16 Aug 2022 04:23:06 GMT
• Title: Knowledge-Injected Federated Learning
• Authors: Zhenan Fan, Zirui Zhou, Jian Pei, Michael P. Friedlander, Jiajie Hu, Chengliang Li, Yong Zhang
• Abstract summary: Federated learning is an emerging technique for training models from decentralized data sets. In many applications, data owners participating in the federated learning system hold not only the data but also a set of domain knowledge. We propose a federated learning framework that allows the injection of participants' domain knowledge.
• Score: 44.89926234630289
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Federated learning is an emerging technique for training models from decentralized data sets. In many applications, data owners participating in the federated learning system hold not only the data but also a set of domain knowledge. Such knowledge includes human know-how and craftsmanship that can be extremely helpful to the federated learning task. In this work, we propose a federated learning framework that allows the injection of participants' domain knowledge, where the key idea is to refine the global model with knowledge locally. The scenario we consider is motivated by a real industry-level application, and we demonstrate the effectiveness of our approach to this application.

Related papers

• Large Language Models are Limited in Out-of-Context Knowledge Reasoning [65.72847298578071]
  Large Language Models (LLMs) possess extensive knowledge and strong capabilities in performing in-context reasoning. This paper focuses on a significant aspect of out-of-context reasoning: Out-of-Context Knowledge Reasoning (OCKR), which is to combine multiple knowledge to infer new knowledge.
  arXiv  Detail & Related papers  (2024-06-11T15:58:59Z)
• KnFu: Effective Knowledge Fusion [5.305607095162403]
  Federated Learning (FL) has emerged as a prominent alternative to the traditional centralized learning approach. The paper proposes Effective Knowledge Fusion (KnFu) algorithm that evaluates knowledge of local models to only fuse semantic neighbors' effective knowledge for each client. A key conclusion of the work is that in scenarios with large and highly heterogeneous local datasets, local training could be preferable to knowledge fusion-based solutions.
  arXiv  Detail & Related papers  (2024-03-18T15:49:48Z)
• Private Knowledge Sharing in Distributed Learning: A Survey [50.51431815732716]
  The rise of Artificial Intelligence has revolutionized numerous industries and transformed the way society operates. It is crucial to utilize information in learning processes that are either distributed or owned by different entities. Modern data-driven services have been developed to integrate distributed knowledge entities into their outcomes.
  arXiv  Detail & Related papers  (2024-02-08T07:18:23Z)
• Exploring Machine Learning Models for Federated Learning: A Review of Approaches, Performance, and Limitations [1.1060425537315088]
  Federated learning is a distributed learning framework enhanced to preserve the privacy of individuals' data. In times of crisis, when real-time decision-making is critical, federated learning allows multiple entities to work collectively without sharing sensitive data. This paper is a systematic review of the literature on privacy-preserving machine learning in the last few years.
  arXiv  Detail & Related papers  (2023-11-17T19:23:21Z)
• Federated Learning: Organizational Opportunities, Challenges, and Adoption Strategies [39.58317527488534]
  Federated learning allows distributed clients to train models collaboratively without the need to share their respective training data with others. We argue that federated learning presents organizational challenges with ample interdisciplinary opportunities for information systems researchers.
  arXiv  Detail & Related papers  (2023-08-04T09:23:23Z)
• UNTER: A Unified Knowledge Interface for Enhancing Pre-trained Language Models [100.4659557650775]
  We propose a UNified knowledge inTERface, UNTER, to provide a unified perspective to exploit both structured knowledge and unstructured knowledge. With both forms of knowledge injected, UNTER gains continuous improvements on a series of knowledge-driven NLP tasks.
  arXiv  Detail & Related papers  (2023-05-02T17:33:28Z)
• Heterogeneous Federated Knowledge Graph Embedding Learning and Unlearning [14.063276595895049]
  Federated Learning (FL) is a paradigm to train a global machine learning model across distributed clients without sharing raw data. We propose FedLU, a novel FL framework for heterogeneous KG embedding learning and unlearning. We show that FedLU achieves superior results in both link prediction and knowledge forgetting.
  arXiv  Detail & Related papers  (2023-02-04T02:44:48Z)
• Knowledge-augmented Deep Learning and Its Applications: A Survey [60.221292040710885]
  knowledge-augmented deep learning (KADL) aims to identify domain knowledge and integrate it into deep models for data-efficient, generalizable, and interpretable deep learning. This survey subsumes existing works and offers a bird's-eye view of research in the general area of knowledge-augmented deep learning.
  arXiv  Detail & Related papers  (2022-11-30T03:44:15Z)
• Knowledge-Aware Federated Active Learning with Non-IID Data [75.98707107158175]
  We propose a federated active learning paradigm to efficiently learn a global model with limited annotation budget. The main challenge faced by federated active learning is the mismatch between the active sampling goal of the global model on the server and that of the local clients. We propose Knowledge-Aware Federated Active Learning (KAFAL), which consists of Knowledge-Specialized Active Sampling (KSAS) and Knowledge-Compensatory Federated Update (KCFU)
  arXiv  Detail & Related papers  (2022-11-24T13:08:43Z)
• Global Knowledge Distillation in Federated Learning [3.7311680121118345]
  We propose a novel global knowledge distillation method, named FedGKD, which learns the knowledge from past global models to tackle down the local bias training problem. To demonstrate the effectiveness of the proposed method, we conduct extensive experiments on various CV datasets (CIFAR-10/100) and settings (non-i.i.d data)
  arXiv  Detail & Related papers  (2021-06-30T18:14:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.