Federated Recommendation via Hybrid Retrieval Augmented Generation
- URL: http://arxiv.org/abs/2403.04256v1
- Date: Thu, 7 Mar 2024 06:38:41 GMT
- Title: Federated Recommendation via Hybrid Retrieval Augmented Generation
- Authors: Huimin Zeng, Zhenrui Yue, Qian Jiang, Dong Wang
- Abstract summary: Federated Recommendation (FR) enables privacy-preserving recommendations.
Large Language Models (LLMs) as recommenders have proven effective across various recommendation scenarios.
We propose GPT-FedRec, a federated recommendation framework leveraging ChatGPT and a novel hybrid Retrieval Augmented Generation (RAG) mechanism.
- Score: 16.228589300933262
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Federated Recommendation (FR) emerges as a novel paradigm that enables
privacy-preserving recommendations. However, traditional FR systems usually
represent users/items with discrete identities (IDs), suffering from
performance degradation due to the data sparsity and heterogeneity in FR. On
the other hand, Large Language Models (LLMs) as recommenders have proven
effective across various recommendation scenarios. Yet, LLM-based recommenders
encounter challenges such as low inference efficiency and potential
hallucination, compromising their performance in real-world scenarios. To this
end, we propose GPT-FedRec, a federated recommendation framework leveraging
ChatGPT and a novel hybrid Retrieval Augmented Generation (RAG) mechanism.
GPT-FedRec is a two-stage solution. The first stage is a hybrid retrieval
process, mining ID-based user patterns and text-based item features. Next, the
retrieved results are converted into text prompts and fed into GPT for
re-ranking. Our proposed hybrid retrieval mechanism and LLM-based re-rank aims
to extract generalized features from data and exploit pretrained knowledge
within LLM, overcoming data sparsity and heterogeneity in FR. In addition, the
RAG approach also prevents LLM hallucination, improving the recommendation
performance for real-world users. Experimental results on diverse benchmark
datasets demonstrate the superior performance of GPT-FedRec against
state-of-the-art baseline methods.
Related papers
- GenRec: A Flexible Data Generator for Recommendations [1.384948712833979]
GenRec is a novel framework for generating synthetic user-item interactions that exhibit realistic and well-known properties.
The framework is based on a generative process based on latent factor modeling.
arXiv Detail & Related papers (2024-07-23T15:53:17Z) - A Thorough Performance Benchmarking on Lightweight Embedding-based Recommender Systems [67.52782366565658]
State-of-the-art recommender systems (RSs) depend on categorical features, which ecoded by embedding vectors, resulting in excessively large embedding tables.
Despite the prosperity of lightweight embedding-based RSs, a wide diversity is seen in evaluation protocols.
This study investigates various LERS' performance, efficiency, and cross-task transferability via a thorough benchmarking process.
arXiv Detail & Related papers (2024-06-25T07:45:00Z) - Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizer [52.09480867526656]
We identify the source of misalignment as a form of distributional shift and uncertainty in learning human preferences.
To mitigate overoptimization, we first propose a theoretical algorithm that chooses the best policy for an adversarially chosen reward model.
Using the equivalence between reward models and the corresponding optimal policy, the algorithm features a simple objective that combines a preference optimization loss and a supervised learning loss.
arXiv Detail & Related papers (2024-05-26T05:38:50Z) - IDGenRec: LLM-RecSys Alignment with Textual ID Learning [48.018397048791115]
We propose IDGen, representing each item as a unique, concise, semantically rich, platform-agnostic textual ID.
We show that IDGen consistently surpasses existing models in sequential recommendation under standard experimental setting.
Results show that the zero-shot performance of the pre-trained foundation model is comparable to or even better than some traditional recommendation models.
arXiv Detail & Related papers (2024-03-27T21:22:37Z) - FedRec+: Enhancing Privacy and Addressing Heterogeneity in Federated
Recommendation Systems [15.463595798992621]
FedRec+ is an ensemble framework for federated recommendation systems.
It enhances privacy and reduces communication costs for edge users.
Experimental results demonstrate the state-of-the-art performance of FedRec+.
arXiv Detail & Related papers (2023-10-31T05:36:53Z) - LlamaRec: Two-Stage Recommendation using Large Language Models for
Ranking [10.671747198171136]
We propose a two-stage framework using large language models for ranking-based recommendation (LlamaRec)
In particular, we use small-scale sequential recommenders to retrieve candidates based on the user interaction history.
LlamaRec consistently achieves datasets superior performance in both recommendation performance and efficiency.
arXiv Detail & Related papers (2023-10-25T06:23:48Z) - Is ChatGPT Fair for Recommendation? Evaluating Fairness in Large
Language Model Recommendation [52.62492168507781]
We propose a novel benchmark called Fairness of Recommendation via LLM (FaiRLLM)
This benchmark comprises carefully crafted metrics and a dataset that accounts for eight sensitive attributes.
By utilizing our FaiRLLM benchmark, we conducted an evaluation of ChatGPT and discovered that it still exhibits unfairness to some sensitive attributes when generating recommendations.
arXiv Detail & Related papers (2023-05-12T16:54:36Z) - Fine-grained Retrieval Prompt Tuning [149.9071858259279]
Fine-grained Retrieval Prompt Tuning steers a frozen pre-trained model to perform the fine-grained retrieval task from the perspectives of sample prompt and feature adaptation.
Our FRPT with fewer learnable parameters achieves the state-of-the-art performance on three widely-used fine-grained datasets.
arXiv Detail & Related papers (2022-07-29T04:10:04Z) - Cluster Regularization via a Hierarchical Feature Regression [0.0]
This paper proposes a novel cluster-based regularization - the hierarchical feature regression (HFR)
It mobilizes insights from the domains of machine learning and graph theory to estimate parameters along a supervised hierarchical representation of the predictor set.
An application to the prediction of economic growth is used to illustrate the HFR's effectiveness in an empirical setting.
arXiv Detail & Related papers (2021-07-10T13:03:01Z) - Reward Constrained Interactive Recommendation with Natural Language
Feedback [158.8095688415973]
We propose a novel constraint-augmented reinforcement learning (RL) framework to efficiently incorporate user preferences over time.
Specifically, we leverage a discriminator to detect recommendations violating user historical preference.
Our proposed framework is general and is further extended to the task of constrained text generation.
arXiv Detail & Related papers (2020-05-04T16:23:34Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.