FedRIR: Rethinking Information Representation in Federated Learning

• URL: http://arxiv.org/abs/2502.00859v1
• Date: Sun, 02 Feb 2025 17:17:29 GMT
• Title: FedRIR: Rethinking Information Representation in Federated Learning
• Authors: Yongqiang Huang, Zerui Shao, Ziyuan Yang, Zexin Lu, Yi Zhang,
• Abstract summary: Mobile and Web-of-Things (WoT) devices at the network edge generate vast amounts of data for machine learning applications. Mobile and Web-of-Things (WoT) devices at the network edge generate vast amounts of data for machine learning applications.
• Score: 7.765917932796046
• License:
• Abstract: Mobile and Web-of-Things (WoT) devices at the network edge generate vast amounts of data for machine learning applications, yet privacy concerns hinder centralized model training. Federated Learning (FL) allows clients (devices) to collaboratively train a shared model coordinated by a central server without transfer private data, but inherent statistical heterogeneity among clients presents challenges, often leading to a dilemma between clients' needs for personalized local models and the server's goal of building a generalized global model. Existing FL methods typically prioritize either global generalization or local personalization, resulting in a trade-off between these two objectives and limiting the full potential of diverse client data. To address this challenge, we propose a novel framework that simultaneously enhances global generalization and local personalization by Rethinking Information Representation in the Federated learning process (FedRIR). Specifically, we introduce Masked Client-Specific Learning (MCSL), which isolates and extracts fine-grained client-specific features tailored to each client's unique data characteristics, thereby enhancing personalization. Concurrently, the Information Distillation Module (IDM) refines the global shared features by filtering out redundant client-specific information, resulting in a purer and more robust global representation that enhances generalization. By integrating the refined global features with the isolated client-specific features, we construct enriched representations that effectively capture both global patterns and local nuances, thereby improving the performance of downstream tasks on the client. The code is available at https://github.com/Deep-Imaging-Group/FedRIR.

Related papers

• An Aggregation-Free Federated Learning for Tackling Data Heterogeneity [50.44021981013037]
  Federated Learning (FL) relies on the effectiveness of utilizing knowledge from distributed datasets. Traditional FL methods adopt an aggregate-then-adapt framework, where clients update local models based on a global model aggregated by the server from the previous training round. We introduce FedAF, a novel aggregation-free FL algorithm.
  arXiv  Detail & Related papers  (2024-04-29T05:55:23Z)
• Federated Deep Multi-View Clustering with Global Self-Supervision [51.639891178519136]
  Federated multi-view clustering has the potential to learn a global clustering model from data distributed across multiple devices. In this setting, label information is unknown and data privacy must be preserved. We propose a novel federated deep multi-view clustering method that can mine complementary cluster structures from multiple clients.
  arXiv  Detail & Related papers  (2023-09-24T17:07:01Z)
• FAM: fast adaptive federated meta-learning [10.980548731600116]
  We propose a fast adaptive federated meta-learning (FAM) framework for collaboratively learning a single global model. A skeleton network is grown on each client to train a personalized model by learning additional client-specific parameters from local data. The personalized client models outperformed the locally trained models, demonstrating the efficacy of the FAM mechanism.
  arXiv  Detail & Related papers  (2023-08-26T22:54:45Z)
• Collaborative Chinese Text Recognition with Personalized Federated Learning [61.34060587461462]
  In Chinese text recognition, it is often necessary for one organization to collect a large amount of data from similar organizations. Due to the natural presence of private information in text data, such as addresses and phone numbers, different organizations are unwilling to share private data. We introduce personalized federated learning (pFL) into the Chinese text recognition task and propose the pFedCR algorithm.
  arXiv  Detail & Related papers  (2023-05-09T16:51:00Z)
• DYNAFED: Tackling Client Data Heterogeneity with Global Dynamics [60.60173139258481]
  Local training on non-iid distributed data results in deflected local optimum. A natural solution is to gather all client data onto the server, such that the server has a global view of the entire data distribution. In this paper, we put forth an idea to collect and leverage global knowledge on the server without hindering data privacy.
  arXiv  Detail & Related papers  (2022-11-20T06:13:06Z)
• Federated Learning in Non-IID Settings Aided by Differentially Private Synthetic Data [20.757477553095637]
  Federated learning (FL) is a privacy-promoting framework that enables clients to collaboratively train machine learning models. A major challenge in federated learning arises when the local data is heterogeneous. We propose FedDPMS, an FL algorithm in which clients deploy variational auto-encoders to augment local datasets with data synthesized using differentially private means of latent data representations.
  arXiv  Detail & Related papers  (2022-06-01T18:00:48Z)
• FRAug: Tackling Federated Learning with Non-IID Features via Representation Augmentation [31.12851987342467]
  Federated Learning (FL) is a decentralized learning paradigm in which multiple clients collaboratively train deep learning models. We propose Federated Representation Augmentation (FRAug) to tackle this practical and challenging problem. Our approach generates synthetic client-specific samples in the embedding space to augment the usually small client datasets.
  arXiv  Detail & Related papers  (2022-05-30T07:43:42Z)
• Personalized Federated Learning through Local Memorization [10.925242558525683]
  Federated learning allows clients to collaboratively learn statistical models while keeping their data local. Recent personalized federated learning methods train a separate model for each client while still leveraging the knowledge available at other clients. We show on a suite of federated datasets that this approach achieves significantly higher accuracy and fairness than state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-11-17T19:40:07Z)
• GRP-FED: Addressing Client Imbalance in Federated Learning via Global-Regularized Personalization [6.592268037926868]
  We present Global-Regularized Personalization (GRP-FED) to tackle the data imbalanced issue. With adaptive aggregation, the global model treats multiple clients fairly and mitigates the global long-tailed issue. Our results show that our GRP-FED improves under both global and local scenarios.
  arXiv  Detail & Related papers  (2021-08-31T14:09:04Z)
• Personalized Federated Learning with First Order Model Optimization [76.81546598985159]
  We propose an alternative to federated learning, where each client federates with other relevant clients to obtain a stronger model per client-specific objectives. We do not assume knowledge of underlying data distributions or client similarities, and allow each client to optimize for arbitrary target distributions of interest. Our method outperforms existing alternatives, while also enabling new features for personalized FL such as transfer outside of local data distributions.
  arXiv  Detail & Related papers  (2020-12-15T19:30:29Z)
• Decentralised Learning from Independent Multi-Domain Labels for Person Re-Identification [69.29602103582782]
  Deep learning has been successful for many computer vision tasks due to the availability of shared and centralised large-scale training data. However, increasing awareness of privacy concerns poses new challenges to deep learning, especially for person re-identification (Re-ID) We propose a novel paradigm called Federated Person Re-Identification (FedReID) to construct a generalisable global model (a central server) by simultaneously learning with multiple privacy-preserved local models (local clients) This client-server collaborative learning process is iteratively performed under privacy control, enabling FedReID to realise decentralised learning without sharing distributed data nor collecting any
  arXiv  Detail & Related papers  (2020-06-07T13:32:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.