Meta Matrix Factorization for Federated Rating Predictions

• URL: http://arxiv.org/abs/1910.10086v4
• Date: Sat, 4 Mar 2023 05:49:19 GMT
• Title: Meta Matrix Factorization for Federated Rating Predictions
• Authors: Yujie Lin, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Dongxiao Yu, Jun Ma, Maarten de Rijke, Xiuzhen Cheng
• Abstract summary: Federated recommender systems have distinct advantages in terms of privacy protection over traditional recommender systems. Previous work on federated recommender systems does not fully consider the limitations of storage, RAM, energy and communication bandwidth in a mobile environment. Our goal in this paper is to design a novel federated learning framework for rating prediction (RP) for mobile environments.
• Score: 84.69112252208468
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Federated recommender systems have distinct advantages in terms of privacy protection over traditional recommender systems that are centralized at a data center. However, previous work on federated recommender systems does not fully consider the limitations of storage, RAM, energy and communication bandwidth in a mobile environment. The scales of the models proposed are too large to be easily run on mobile devices. And existing federated recommender systems need to fine-tune recommendation models on each device, making it hard to effectively exploit collaborative filtering information among users/devices. Our goal in this paper is to design a novel federated learning framework for rating prediction (RP) for mobile environments. We introduce a federated matrix factorization (MF) framework, named meta matrix factorization (MetaMF). Given a user, we first obtain a collaborative vector by collecting useful information with a collaborative memory module. Then, we employ a meta recommender module to generate private item embeddings and a RP model based on the collaborative vector in the server. To address the challenge of generating a large number of high-dimensional item embeddings, we devise a rise-dimensional generation strategy that first generates a low-dimensional item embedding matrix and a rise-dimensional matrix, and then multiply them to obtain high-dimensional embeddings. We use the generated model to produce private RPs for the given user on her device. MetaMF shows a high capacity even with a small RP model, which can adapt to the limitations of a mobile environment. We conduct extensive experiments on four benchmark datasets to compare MetaMF with existing MF methods and find that MetaMF can achieve competitive performance. Moreover, we find MetaMF achieves higher RP performance over existing federated methods by better exploiting collaborative filtering among users/devices.

Related papers

• Read-ME: Refactorizing LLMs as Router-Decoupled Mixture of Experts with System Co-Design [59.00758127310582]
  We propose a novel framework Read-ME that transforms pre-trained dense LLMs into smaller MoE models. Our approach employs activation sparsity to extract experts. Read-ME outperforms other popular open-source dense models of similar scales.
  arXiv  Detail & Related papers  (2024-10-24T19:48:51Z)
• Personalized Item Representations in Federated Multimodal Recommendation [37.52127488593226]
  Federated Multimodal Recommendation System, called FedMR, encodes multimodal item data. FedMR is compatible with existing ID-based federated recommendation systems. Experiments on four real-world multimodal datasets demonstrate FedMR's effectiveness.
  arXiv  Detail & Related papers  (2024-10-11T03:10:09Z)
• Improved Diversity-Promoting Collaborative Metric Learning for Recommendation [127.08043409083687]
  Collaborative Metric Learning (CML) has recently emerged as a popular method in recommendation systems. This paper focuses on a challenging scenario where a user has multiple categories of interests. We propose a novel method called textitDiversity-Promoting Collaborative Metric Learning (DPCML)
  arXiv  Detail & Related papers  (2024-09-02T07:44:48Z)
• Personalized Federated Collaborative Filtering: A Variational AutoEncoder Approach [49.63614966954833]
  Federated Collaborative Filtering (FedCF) is an emerging field focused on developing a new recommendation framework with preserving privacy. This paper proposes a novel personalized FedCF method by preserving users' personalized information into a latent variable and a neural model simultaneously. To effectively train the proposed framework, we model the problem as a specialized Variational AutoEncoder (VAE) task by integrating user interaction vector reconstruction with missing value prediction.
  arXiv  Detail & Related papers  (2024-08-16T05:49:14Z)
• (Amplified) Banded Matrix Factorization: A unified approach to private training [15.922315074913255]
  Matrix factorization (MF) mechanisms for differential privacy (DP) have substantially improved the state-of-the-art in privacy-utility-computation tradeoffs for ML applications. We show how MF can subsume prior state-of-the-art algorithms in both federated and centralized training settings.
  arXiv  Detail & Related papers  (2023-06-13T21:53:17Z)
• Federated Privacy-preserving Collaborative Filtering for On-Device Next App Prediction [52.16923290335873]
  We propose a novel SeqMF model to solve the problem of predicting the next app launch during mobile device usage. We modify the structure of the classical matrix factorization model and update the training procedure to sequential learning. One more ingredient of the proposed approach is a new privacy mechanism that guarantees the protection of the sent data from the users to the remote server.
  arXiv  Detail & Related papers  (2023-02-05T10:29:57Z)
• PrivMVMF: Privacy-Preserving Multi-View Matrix Factorization for Recommender Systems [0.190365714903665]
  We propose a new privacy-preserving framework based on homomorphic encryption, Privacy-Preserving Multi-View Matrix Factorization (PrivMVMF) PrivMVMF is successfully implemented and tested thoroughly with the MovieLens dataset.
  arXiv  Detail & Related papers  (2022-09-29T03:21:24Z)
• FedMix: Approximation of Mixup under Mean Augmented Federated Learning [60.503258658382]
  Federated learning (FL) allows edge devices to collectively learn a model without directly sharing data within each device. Current state-of-the-art algorithms suffer from performance degradation as the heterogeneity of local data across clients increases. We propose a new augmentation algorithm, named FedMix, which is inspired by a phenomenal yet simple data augmentation method, Mixup.
  arXiv  Detail & Related papers  (2021-07-01T06:14:51Z)
• FedNS: Improving Federated Learning for collaborative image classification on mobile clients [22.980223900446997]
  Federated Learning (FL) is a paradigm that aims to support loosely connected clients in learning a global model. We propose a new approach, termed Federated Node Selection (FedNS), for the server's global model aggregation in the FL setting. We show with experiments from multiple datasets and networks that FedNS can consistently achieve improved performance over FedAvg.
  arXiv  Detail & Related papers  (2021-01-20T06:45:46Z)
• Evaluation Framework For Large-scale Federated Learning [10.127616622630514]
  Federated learning is proposed as a machine learning setting to enable distributed edge devices, such as mobile phones, to collaboratively learn a shared prediction model. In this paper, we introduce a framework designed for large-scale federated learning which consists of approaches to generating dataset and modular evaluation framework.
  arXiv  Detail & Related papers  (2020-03-03T15:12:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.