Fed2: Feature-Aligned Federated Learning
- URL: http://arxiv.org/abs/2111.14248v1
- Date: Sun, 28 Nov 2021 22:21:48 GMT
- Title: Fed2: Feature-Aligned Federated Learning
- Authors: Fuxun Yu, Weishan Zhang, Zhuwei Qin, Zirui Xu, Di Wang, Chenchen Liu,
Zhi Tian, Xiang Chen
- Abstract summary: Federated learning learns from scattered data by fusing collaborative models from local nodes.
We propose Fed2, a feature-aligned federated learning framework to resolve this issue by establishing a firm structure-feature alignment.
Fed2 could effectively enhance the federated learning convergence performance under extensive homo- and heterogeneous settings.
- Score: 32.54574459692627
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Federated learning learns from scattered data by fusing collaborative models
from local nodes. However, the conventional coordinate-based model averaging by
FedAvg ignored the random information encoded per parameter and may suffer from
structural feature misalignment. In this work, we propose Fed2, a
feature-aligned federated learning framework to resolve this issue by
establishing a firm structure-feature alignment across the collaborative
models. Fed2 is composed of two major designs: First, we design a
feature-oriented model structure adaptation method to ensure explicit feature
allocation in different neural network structures. Applying the structure
adaptation to collaborative models, matchable structures with similar feature
information can be initialized at the very early training stage. During the
federated learning process, we then propose a feature paired averaging scheme
to guarantee aligned feature distribution and maintain no feature fusion
conflicts under either IID or non-IID scenarios. Eventually, Fed2 could
effectively enhance the federated learning convergence performance under
extensive homo- and heterogeneous settings, providing excellent convergence
speed, accuracy, and computation/communication efficiency.
Related papers
- Learn What You Need in Personalized Federated Learning [53.83081622573734]
$textitLearn2pFed$ is a novel algorithm-unrolling-based personalized federated learning framework.
We show that $textitLearn2pFed$ significantly outperforms previous personalized federated learning methods.
arXiv Detail & Related papers (2024-01-16T12:45:15Z) - Fed-CO2: Cooperation of Online and Offline Models for Severe Data
Heterogeneity in Federated Learning [14.914477928398133]
Federated Learning (FL) has emerged as a promising distributed learning paradigm.
The effectiveness of FL is highly dependent on the quality of the data that is being used for training.
We propose Fed-CO$_2$, a universal FL framework that handles both label distribution skew and feature skew.
arXiv Detail & Related papers (2023-12-21T15:12:12Z) - Factor-Assisted Federated Learning for Personalized Optimization with
Heterogeneous Data [6.024145412139383]
Federated learning is an emerging distributed machine learning framework aiming at protecting data privacy.
Data in different clients contain both common knowledge and personalized knowledge.
We develop a novel personalized federated learning framework for heterogeneous data, which we refer to as FedSplit.
arXiv Detail & Related papers (2023-12-07T13:05:47Z) - Faster Adaptive Federated Learning [84.38913517122619]
Federated learning has attracted increasing attention with the emergence of distributed data.
In this paper, we propose an efficient adaptive algorithm (i.e., FAFED) based on momentum-based variance reduced technique in cross-silo FL.
arXiv Detail & Related papers (2022-12-02T05:07:50Z) - FedX: Unsupervised Federated Learning with Cross Knowledge Distillation [70.73383643450548]
FedX is an unsupervised federated learning framework.
It learns unbiased representation from decentralized and heterogeneous local data.
It can be used as an add-on module for existing unsupervised algorithms in federated settings.
arXiv Detail & Related papers (2022-07-19T09:56:44Z) - AFAFed -- Protocol analysis [3.016628653955123]
This is a novel A Fair Federated Adaptive learning framework for stream-oriented IoT application environments.
We analyze the convergence properties and address the implementation aspects AFAFed.
arXiv Detail & Related papers (2022-06-29T22:12:08Z) - Disentangled Federated Learning for Tackling Attributes Skew via
Invariant Aggregation and Diversity Transferring [104.19414150171472]
Attributes skews the current federated learning (FL) frameworks from consistent optimization directions among the clients.
We propose disentangled federated learning (DFL) to disentangle the domain-specific and cross-invariant attributes into two complementary branches.
Experiments verify that DFL facilitates FL with higher performance, better interpretability, and faster convergence rate, compared with SOTA FL methods.
arXiv Detail & Related papers (2022-06-14T13:12:12Z) - Semi-supervised Domain Adaptive Structure Learning [72.01544419893628]
Semi-supervised domain adaptation (SSDA) is a challenging problem requiring methods to overcome both 1) overfitting towards poorly annotated data and 2) distribution shift across domains.
We introduce an adaptive structure learning method to regularize the cooperation of SSL and DA.
arXiv Detail & Related papers (2021-12-12T06:11:16Z) - Robustness and Personalization in Federated Learning: A Unified Approach
via Regularization [4.7234844467506605]
We present a class of methods for robust, personalized federated learning, called Fed+.
The principal advantage of Fed+ is to better accommodate the real-world characteristics found in federated training.
We demonstrate the benefits of Fed+ through extensive experiments on benchmark datasets.
arXiv Detail & Related papers (2020-09-14T10:04:30Z) - Heterogeneous Federated Learning [41.04946606973614]
Federated learning learns from scattered data by fusing collaborative models from local nodes.
Due to chaotic information distribution, the model fusion may suffer from structural misalignment with regard to unmatched parameters.
We propose a novel federated learning framework to establish a firm structure-information alignment across collaborative models.
arXiv Detail & Related papers (2020-08-15T19:06:59Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.