FedSPLIT: One-Shot Federated Recommendation System Based on Non-negative
Joint Matrix Factorization and Knowledge Distillation
- URL: http://arxiv.org/abs/2205.02359v1
- Date: Wed, 4 May 2022 23:42:14 GMT
- Title: FedSPLIT: One-Shot Federated Recommendation System Based on Non-negative
Joint Matrix Factorization and Knowledge Distillation
- Authors: Maksim E. Eren, Luke E. Richards, Manish Bhattarai, Roberto Yus,
Charles Nicholas, Boian S. Alexandrov
- Abstract summary: We present the first unsupervised one-shot federated CF implementation, named FedSPLIT, based on NMF joint factorization.
FedSPLIT can obtain similar results than the state of the art (and even outperform it in certain situations) with a substantial decrease in the number of communications.
- Score: 7.621960305708476
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Non-negative matrix factorization (NMF) with missing-value completion is a
well-known effective Collaborative Filtering (CF) method used to provide
personalized user recommendations. However, traditional CF relies on the
privacy-invasive collection of users' explicit and implicit feedback to build a
central recommender model. One-shot federated learning has recently emerged as
a method to mitigate the privacy problem while addressing the traditional
communication bottleneck of federated learning. In this paper, we present the
first unsupervised one-shot federated CF implementation, named FedSPLIT, based
on NMF joint factorization. In our solution, the clients first apply local CF
in-parallel to build distinct client-specific recommenders. Then, the
privacy-preserving local item patterns and biases from each client are shared
with the processor to perform joint factorization in order to extract the
global item patterns. Extracted patterns are then aggregated to each client to
build the local models via knowledge distillation. In our experiments, we
demonstrate the feasibility of our approach with standard recommendation
datasets. FedSPLIT can obtain similar results than the state of the art (and
even outperform it in certain situations) with a substantial decrease in the
number of communications.
Related papers
- Beyond Similarity: Personalized Federated Recommendation with Composite Aggregation [22.359428566363945]
Federated recommendation aims to collect global knowledge by aggregating local models from massive devices.
Current methods mainly leverage aggregation functions invented by federated vision community to aggregate parameters from similar clients.
We propose a personalized Federated recommendation model with Composite Aggregation (FedCA)
arXiv Detail & Related papers (2024-06-06T10:17:52Z) - FedJETs: Efficient Just-In-Time Personalization with Federated Mixture
of Experts [48.78037006856208]
FedJETs is a novel solution by using a Mixture-of-Experts (MoE) framework within a Federated Learning (FL) setup.
Our method leverages the diversity of the clients to train specialized experts on different subsets of classes, and a gating function to route the input to the most relevant expert(s)
Our approach can improve accuracy up to 18% in state of the art FL settings, while maintaining competitive zero-shot performance.
arXiv Detail & Related papers (2023-06-14T15:47:52Z) - Personalized Federated Learning under Mixture of Distributions [98.25444470990107]
We propose a novel approach to Personalized Federated Learning (PFL), which utilizes Gaussian mixture models (GMM) to fit the input data distributions across diverse clients.
FedGMM possesses an additional advantage of adapting to new clients with minimal overhead, and it also enables uncertainty quantification.
Empirical evaluations on synthetic and benchmark datasets demonstrate the superior performance of our method in both PFL classification and novel sample detection.
arXiv Detail & Related papers (2023-05-01T20:04:46Z) - User-Centric Federated Learning: Trading off Wireless Resources for
Personalization [18.38078866145659]
In Federated Learning (FL) systems, Statistical Heterogeneousness increases the algorithm convergence time and reduces the generalization performance.
To tackle the above problems without violating the privacy constraints that FL imposes, personalized FL methods have to couple statistically similar clients without directly accessing their data.
In this work, we design user-centric aggregation rules that are based on readily available gradient information and are capable of producing personalized models for each FL client.
Our algorithm outperforms popular personalized FL baselines in terms of average accuracy, worst node performance, and training communication overhead.
arXiv Detail & Related papers (2023-04-25T15:45:37Z) - PGFed: Personalize Each Client's Global Objective for Federated Learning [7.810284483002312]
We propose a novel personalized FL framework that enables each client to personalize its own global objective.
To avoid massive (O(N2)) communication overhead and potential privacy leakage, each client's risk is estimated through a first-order approximation for other clients' adaptive risk aggregation.
Our experiments on four datasets under different federated settings show consistent improvements of PGFed over previous state-of-the-art methods.
arXiv Detail & Related papers (2022-12-02T21:16:39Z) - User-Specific Bicluster-based Collaborative Filtering: Handling
Preference Locality, Sparsity and Subjectivity [1.0398909602421018]
Collaborative Filtering (CF) is the most common approach to build Recommender Systems.
We propose USBFC, a Biclustering-based CF approach that creates user-specific models from strongly coherent and statistically significant rating patterns.
USBFC achieves competitive predictive accuracy against state-of-the-art CF methods.
arXiv Detail & Related papers (2022-11-15T18:10:52Z) - Anomaly Detection through Unsupervised Federated Learning [0.0]
Federated learning is proving to be one of the most promising paradigms for leveraging distributed resources.
We propose a novel method in which, through a preprocessing phase, clients are grouped into communities.
The resulting anomaly detection model is then shared and used to detect anomalies within the clients of the same community.
arXiv Detail & Related papers (2022-09-09T08:45:47Z) - FedCL: Federated Contrastive Learning for Privacy-Preserving
Recommendation [98.5705258907774]
FedCL can exploit high-quality negative samples for effective model training with privacy well protected.
We first infer user embeddings from local user data through the local model on each client, and then perturb them with local differential privacy (LDP)
Since individual user embedding contains heavy noise due to LDP, we propose to cluster user embeddings on the server to mitigate the influence of noise.
arXiv Detail & Related papers (2022-04-21T02:37:10Z) - On the Convergence of Clustered Federated Learning [57.934295064030636]
In a federated learning system, the clients, e.g. mobile devices and organization participants, usually have different personal preferences or behavior patterns.
This paper proposes a novel weighted client-based clustered FL algorithm to leverage the client's group and each client in a unified optimization framework.
arXiv Detail & Related papers (2022-02-13T02:39:19Z) - Toward Understanding the Influence of Individual Clients in Federated
Learning [52.07734799278535]
Federated learning allows clients to jointly train a global model without sending their private data to a central server.
We defined a new notion called em-Influence, quantify this influence over parameters, and proposed an effective efficient model to estimate this metric.
arXiv Detail & Related papers (2020-12-20T14:34:36Z) - Personalized Federated Learning with First Order Model Optimization [76.81546598985159]
We propose an alternative to federated learning, where each client federates with other relevant clients to obtain a stronger model per client-specific objectives.
We do not assume knowledge of underlying data distributions or client similarities, and allow each client to optimize for arbitrary target distributions of interest.
Our method outperforms existing alternatives, while also enabling new features for personalized FL such as transfer outside of local data distributions.
arXiv Detail & Related papers (2020-12-15T19:30:29Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.