Language Models Encode Collaborative Signals in Recommendation

• URL: http://arxiv.org/abs/2407.05441v1
• Date: Sun, 7 Jul 2024 17:05:24 GMT
• Title: Language Models Encode Collaborative Signals in Recommendation
• Authors: Leheng Sheng, An Zhang, Yi Zhang, Yuxin Chen, Xiang Wang, Tat-Seng Chua,
• Abstract summary: We show that item representations, when linearly mapped from advanced LM representations, yield superior recommendation performance. Motivated by these findings, we propose a simple yet effective collaborative filtering (CF) model named AlphaRec. AlphaRec is comprised of three main components: a multilayer perceptron (MLP), graph convolution, and contrastive learning (CL) loss function.
• Score: 57.90679739598295
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent studies empirically indicate that language models (LMs) encode rich world knowledge beyond mere semantics, attracting significant attention across various fields. However, in the recommendation domain, it remains uncertain whether LMs implicitly encode user preference information. Contrary to the prevailing understanding that LMs and traditional recommender models learn two distinct representation spaces due to a huge gap in language and behavior modeling objectives, this work rethinks such understanding and explores extracting a recommendation space directly from the language representation space. Surprisingly, our findings demonstrate that item representations, when linearly mapped from advanced LM representations, yield superior recommendation performance. This outcome suggests the homomorphism between the language representation space and an effective recommendation space, implying that collaborative signals may indeed be encoded within advanced LMs. Motivated by these findings, we propose a simple yet effective collaborative filtering (CF) model named AlphaRec, which utilizes language representations of item textual metadata (e.g., titles) instead of traditional ID-based embeddings. Specifically, AlphaRec is comprised of three main components: a multilayer perceptron (MLP), graph convolution, and contrastive learning (CL) loss function, making it extremely easy to implement and train. Our empirical results show that AlphaRec outperforms leading ID-based CF models on multiple datasets, marking the first instance of such a recommender with text embeddings achieving this level of performance. Moreover, AlphaRec introduces a new language-representation-based CF paradigm with several desirable advantages: being easy to implement, lightweight, rapid convergence, superior zero-shot recommendation abilities in new domains, and being aware of user intention.

Related papers

• Training Large Recommendation Models via Graph-Language Token Alignment [53.3142545812349]
  We propose a novel framework to train Large Recommendation models via Graph-Language Token Alignment. By aligning item and user nodes from the interaction graph with pretrained LLM tokens, GLTA effectively leverages the reasoning abilities of LLMs. Furthermore, we introduce Graph-Language Logits Matching (GLLM) to optimize token alignment for end-to-end item prediction.
  arXiv  Detail & Related papers  (2025-02-26T02:19:10Z)
• Boosting the Capabilities of Compact Models in Low-Data Contexts with Large Language Models and Retrieval-Augmented Generation [2.9921619703037274]
  We propose a retrieval augmented generation (RAG) framework backed by a large language model (LLM) to correct the output of a smaller model for the linguistic task of morphological glossing. We leverage linguistic information to make up for the lack of data and trainable parameters, while allowing for inputs from written descriptive grammars interpreted and distilled through an LLM. We show that a compact, RAG-supported model is highly effective in data-scarce settings, achieving a new state-of-the-art for this task and our target languages.
  arXiv  Detail & Related papers  (2024-10-01T04:20:14Z)
• Multi-modal Instruction Tuned LLMs with Fine-grained Visual Perception [63.03288425612792]
  We propose bfAnyRef, a general MLLM model that can generate pixel-wise object perceptions and natural language descriptions from multi-modality references. Our model achieves state-of-the-art results across multiple benchmarks, including diverse modality referring segmentation and region-level referring expression generation.
  arXiv  Detail & Related papers  (2024-03-05T13:45:46Z)
• Unlocking the Potential of Large Language Models for Explainable Recommendations [55.29843710657637]
  It remains uncertain what impact replacing the explanation generator with the recently emerging large language models (LLMs) would have. In this study, we propose LLMXRec, a simple yet effective two-stage explainable recommendation framework. By adopting several key fine-tuning techniques, controllable and fluent explanations can be well generated.
  arXiv  Detail & Related papers  (2023-12-25T09:09:54Z)
• RecExplainer: Aligning Large Language Models for Explaining Recommendation Models [50.74181089742969]
  Large language models (LLMs) have demonstrated remarkable intelligence in understanding, reasoning, and instruction following. This paper presents the initial exploration of using LLMs as surrogate models to explain black-box recommender models. To facilitate an effective alignment, we introduce three methods: behavior alignment, intention alignment, and hybrid alignment.
  arXiv  Detail & Related papers  (2023-11-18T03:05:43Z)
• Reformulating Sequential Recommendation: Learning Dynamic User Interest with Content-enriched Language Modeling [18.297332953450514]
  We propose LANCER, which leverages the semantic understanding capabilities of pre-trained language models to generate personalized recommendations. Our approach bridges the gap between language models and recommender systems, resulting in more human-like recommendations.
  arXiv  Detail & Related papers  (2023-09-19T08:54:47Z)
• Exploring Large Language Model for Graph Data Understanding in Online Job Recommendations [63.19448893196642]
  We present a novel framework that harnesses the rich contextual information and semantic representations provided by large language models to analyze behavior graphs. By leveraging this capability, our framework enables personalized and accurate job recommendations for individual users.
  arXiv  Detail & Related papers  (2023-07-10T11:29:41Z)
• Pre-Trained Language Models for Interactive Decision-Making [72.77825666035203]
  We describe a framework for imitation learning in which goals and observations are represented as a sequence of embeddings. We demonstrate that this framework enables effective generalization across different environments. For test tasks involving novel goals or novel scenes, initializing policies with language models improves task completion rates by 43.6%.
  arXiv  Detail & Related papers  (2022-02-03T18:55:52Z)
• Incorporating Linguistic Knowledge for Abstractive Multi-document Summarization [20.572283625521784]
  We develop a neural network based abstractive multi-document summarization (MDS) model. We process the dependency information into the linguistic-guided attention mechanism. With the help of linguistic signals, sentence-level relations can be correctly captured.
  arXiv  Detail & Related papers  (2021-09-23T08:13:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.