Federated Adaptation for Foundation Model-based Recommendations

• URL: http://arxiv.org/abs/2405.04840v1
• Date: Wed, 08 May 2024 06:27:07 GMT
• Title: Federated Adaptation for Foundation Model-based Recommendations
• Authors: Chunxu Zhang, Guodong Long, Hongkuan Guo, Xiao Fang, Yang Song, Zhaojie Liu, Guorui Zhou, Zijian Zhang, Yang Liu, Bo Yang,
• Abstract summary: We propose a novel adaptation mechanism to enhance the foundation model-based recommendation system in a privacy-preserving manner. User's private behavioral data remains secure as it is not shared with the server. Experimental results on four benchmark datasets demonstrate our method's superior performance.
• Score: 29.86114788739202
• License:
• Abstract: With the recent success of large language models, particularly foundation models with generalization abilities, applying foundation models for recommendations becomes a new paradigm to improve existing recommendation systems. It becomes a new open challenge to enable the foundation model to capture user preference changes in a timely manner with reasonable communication and computation costs while preserving privacy. This paper proposes a novel federated adaptation mechanism to enhance the foundation model-based recommendation system in a privacy-preserving manner. Specifically, each client will learn a lightweight personalized adapter using its private data. The adapter then collaborates with pre-trained foundation models to provide recommendation service efficiently with fine-grained manners. Importantly, users' private behavioral data remains secure as it is not shared with the server. This data localization-based privacy preservation is embodied via the federated learning framework. The model can ensure that shared knowledge is incorporated into all adapters while simultaneously preserving each user's personal preferences. Experimental results on four benchmark datasets demonstrate our method's superior performance. Implementation code is available to ease reproducibility.

Related papers

• Look Back for More: Harnessing Historical Sequential Updates for Personalized Federated Adapter Tuning [50.45027483522507]
  Existing personalized federated learning (PFL) approaches rely solely on the clients' latest updated models. We propose pFedSeq, designed for personalizing adapters to fine-tune a foundation model in FL. In pFedSeq, the server maintains and trains a sequential learner, which processes a sequence of past adapter updates from clients. To effectively capture the cross-client and cross-step relations hidden in previous updates, pFedSeq adopts the powerful selective state space model.
  arXiv  Detail & Related papers  (2025-01-03T06:10:09Z)
• Efficient and Robust Regularized Federated Recommendation [52.24782464815489]
  The recommender system (RSRS) addresses both user preference and privacy concerns. We propose a novel method that incorporates non-uniform gradient descent to improve communication efficiency. RFRecF's superior robustness compared to diverse baselines.
  arXiv  Detail & Related papers  (2024-11-03T12:10:20Z)
• Personalized Federated Collaborative Filtering: A Variational AutoEncoder Approach [49.63614966954833]
  Federated Collaborative Filtering (FedCF) is an emerging field focused on developing a new recommendation framework with preserving privacy. Existing FedCF methods typically combine distributed Collaborative Filtering (CF) algorithms with privacy-preserving mechanisms, and then preserve personalized information into a user embedding vector. This paper proposes a novel personalized FedCF method by preserving users' personalized information into a latent variable and a neural model simultaneously.
  arXiv  Detail & Related papers  (2024-08-16T05:49:14Z)
• Personalized Federated Learning via Stacking [0.0]
  We present a novel personalization approach based on stacked generalization where clients directly send each other privacy-preserving models to be used as base models to train a meta-model on private data. Our approach is flexible, accommodating various privacy-preserving techniques and model types, and can be applied in horizontal, hybrid, and vertically partitioned federations.
  arXiv  Detail & Related papers  (2024-04-16T23:47:23Z)
• User Consented Federated Recommender System Against Personalized Attribute Inference Attack [55.24441467292359]
  We propose a user-consented federated recommendation system (UC-FedRec) to flexibly satisfy the different privacy needs of users. UC-FedRec allows users to self-define their privacy preferences to meet various demands and makes recommendations with user consent.
  arXiv  Detail & Related papers  (2023-12-23T09:44:57Z)
• Dual Personalization on Federated Recommendation [50.4115315992418]
  Federated recommendation is a new Internet service architecture that aims to provide privacy-preserving recommendation services in federated settings. This paper proposes a novel Personalized Federated Recommendation (PFedRec) framework to learn many user-specific lightweight models. We also propose a new dual personalization mechanism to effectively learn fine-grained personalization on both users and items.
  arXiv  Detail & Related papers  (2023-01-16T05:26:07Z)
• PFA: Privacy-preserving Federated Adaptation for Effective Model Personalization [6.66389628571674]
  Federated learning (FL) has become a prevalent distributed machine learning paradigm with improved privacy. This paper introduces a new concept called federated adaptation, targeting at adapting the trained model in a federated manner to achieve better personalization results. We propose PFA, a framework to accomplish Privacy-preserving Federated Adaptation.
  arXiv  Detail & Related papers  (2021-03-02T08:07:34Z)
• Unsupervised Model Personalization while Preserving Privacy and Scalability: An Open Problem [55.21502268698577]
  This work investigates the task of unsupervised model personalization, adapted to continually evolving, unlabeled local user images. We provide a novel Dual User-Adaptation framework (DUA) to explore the problem. This framework flexibly disentangles user-adaptation into model personalization on the server and local data regularization on the user device.
  arXiv  Detail & Related papers  (2020-03-30T09:35:12Z)
• Federating Recommendations Using Differentially Private Prototypes [16.29544153550663]
  We propose a new federated approach to learning global and local private models for recommendation without collecting raw data. By requiring only two rounds of communication, we both reduce the communication costs and avoid the excessive privacy loss. We show local adaptation of the global model allows our method to outperform centralized matrix-factorization-based recommender system models.
  arXiv  Detail & Related papers  (2020-03-01T22:21:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.