Knowledge-Aware Federated Active Learning with Non-IID Data
- URL: http://arxiv.org/abs/2211.13579v3
- Date: Sat, 30 Sep 2023 04:41:21 GMT
- Title: Knowledge-Aware Federated Active Learning with Non-IID Data
- Authors: Yu-Tong Cao, Ye Shi, Baosheng Yu, Jingya Wang, Dacheng Tao
- Abstract summary: We propose a federated active learning paradigm to efficiently learn a global model with limited annotation budget.
The main challenge faced by federated active learning is the mismatch between the active sampling goal of the global model on the server and that of the local clients.
We propose Knowledge-Aware Federated Active Learning (KAFAL), which consists of Knowledge-Specialized Active Sampling (KSAS) and Knowledge-Compensatory Federated Update (KCFU)
- Score: 75.98707107158175
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Federated learning enables multiple decentralized clients to learn
collaboratively without sharing the local training data. However, the expensive
annotation cost to acquire data labels on local clients remains an obstacle in
utilizing local data. In this paper, we propose a federated active learning
paradigm to efficiently learn a global model with limited annotation budget
while protecting data privacy in a decentralized learning way. The main
challenge faced by federated active learning is the mismatch between the active
sampling goal of the global model on the server and that of the asynchronous
local clients. This becomes even more significant when data is distributed
non-IID across local clients. To address the aforementioned challenge, we
propose Knowledge-Aware Federated Active Learning (KAFAL), which consists of
Knowledge-Specialized Active Sampling (KSAS) and Knowledge-Compensatory
Federated Update (KCFU). KSAS is a novel active sampling method tailored for
the federated active learning problem. It deals with the mismatch challenge by
sampling actively based on the discrepancies between local and global models.
KSAS intensifies specialized knowledge in local clients, ensuring the sampled
data to be informative for both the local clients and the global model. KCFU,
in the meantime, deals with the client heterogeneity caused by limited data and
non-IID data distributions. It compensates for each client's ability in weak
classes by the assistance of the global model. Extensive experiments and
analyses are conducted to show the superiority of KSAS over the
state-of-the-art active learning methods and the efficiency of KCFU under the
federated active learning framework.
Related papers
- ConDa: Fast Federated Unlearning with Contribution Dampening [46.074452659791575]
ConDa is a framework that performs efficient unlearning by tracking down the parameters which affect the global model for each client.
We perform experiments on multiple datasets and demonstrate that ConDa is effective to forget a client's data.
arXiv Detail & Related papers (2024-10-05T12:45:35Z) - SFedCA: Credit Assignment-Based Active Client Selection Strategy for Spiking Federated Learning [15.256986486372407]
Spiking federated learning allows resource-constrained devices to train collaboratively at low power consumption without exchanging local data.
Existing spiking federated learning methods employ a random selection approach for client aggregation, assuming unbiased client participation.
We propose a credit assignment-based active client selection strategy, the SFedCA, to judiciously aggregate clients that contribute to the global sample distribution balance.
arXiv Detail & Related papers (2024-06-18T01:56:22Z) - FedAL: Black-Box Federated Knowledge Distillation Enabled by Adversarial Learning [10.118046070458488]
We propose Federated knowledge distillation enabled by Adversarial Learning (FedAL) to address the data heterogeneity among clients.
First, to alleviate the local model output divergence across clients caused by data heterogeneity, the server acts as a discriminator to guide clients' local model training.
We design the less-forgetting regularization for both local training and global knowledge transfer to guarantee clients' ability to transfer/learn knowledge to/from others.
arXiv Detail & Related papers (2023-11-28T08:01:43Z) - FLIS: Clustered Federated Learning via Inference Similarity for Non-IID
Data Distribution [7.924081556869144]
We present a new algorithm, FLIS, which groups the clients population in clusters with jointly trainable data distributions.
We present experimental results to demonstrate the benefits of FLIS over the state-of-the-art benchmarks on CIFAR-100/10, SVHN, and FMNIST datasets.
arXiv Detail & Related papers (2022-08-20T22:10:48Z) - FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling
and Correction [48.85303253333453]
Federated learning (FL) allows multiple clients to collectively train a high-performance global model without sharing their private data.
We propose a novel federated learning algorithm with local drift decoupling and correction (FedDC)
Our FedDC only introduces lightweight modifications in the local training phase, in which each client utilizes an auxiliary local drift variable to track the gap between the local model parameter and the global model parameters.
Experiment results and analysis demonstrate that FedDC yields expediting convergence and better performance on various image classification tasks.
arXiv Detail & Related papers (2022-03-22T14:06:26Z) - Distributed Unsupervised Visual Representation Learning with Fused
Features [13.935997509072669]
Federated learning (FL) enables distributed clients to learn a shared model for prediction while keeping the training data local on each client.
We propose a federated contrastive learning framework consisting of two approaches: feature fusion and neighborhood matching.
It outperforms other methods by 11% on IID data and matches the performance of centralized learning.
arXiv Detail & Related papers (2021-11-21T08:36:31Z) - Towards Fair Federated Learning with Zero-Shot Data Augmentation [123.37082242750866]
Federated learning has emerged as an important distributed learning paradigm, where a server aggregates a global model from many client-trained models while having no access to the client data.
We propose a novel federated learning system that employs zero-shot data augmentation on under-represented data to mitigate statistical heterogeneity and encourage more uniform accuracy performance across clients in federated networks.
We study two variants of this scheme, Fed-ZDAC (federated learning with zero-shot data augmentation at the clients) and Fed-ZDAS (federated learning with zero-shot data augmentation at the server).
arXiv Detail & Related papers (2021-04-27T18:23:54Z) - Exploiting Shared Representations for Personalized Federated Learning [54.65133770989836]
We propose a novel federated learning framework and algorithm for learning a shared data representation across clients and unique local heads for each client.
Our algorithm harnesses the distributed computational power across clients to perform many local-updates with respect to the low-dimensional local parameters for every update of the representation.
This result is of interest beyond federated learning to a broad class of problems in which we aim to learn a shared low-dimensional representation among data distributions.
arXiv Detail & Related papers (2021-02-14T05:36:25Z) - WAFFLe: Weight Anonymized Factorization for Federated Learning [88.44939168851721]
In domains where data are sensitive or private, there is great value in methods that can learn in a distributed manner without the data ever leaving the local devices.
We propose Weight Anonymized Factorization for Federated Learning (WAFFLe), an approach that combines the Indian Buffet Process with a shared dictionary of weight factors for neural networks.
arXiv Detail & Related papers (2020-08-13T04:26:31Z) - Decentralised Learning from Independent Multi-Domain Labels for Person
Re-Identification [69.29602103582782]
Deep learning has been successful for many computer vision tasks due to the availability of shared and centralised large-scale training data.
However, increasing awareness of privacy concerns poses new challenges to deep learning, especially for person re-identification (Re-ID)
We propose a novel paradigm called Federated Person Re-Identification (FedReID) to construct a generalisable global model (a central server) by simultaneously learning with multiple privacy-preserved local models (local clients)
This client-server collaborative learning process is iteratively performed under privacy control, enabling FedReID to realise decentralised learning without sharing distributed data nor collecting any
arXiv Detail & Related papers (2020-06-07T13:32:33Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.