Enhancing Recommender Systems Using Textual Embeddings from Pre-trained Language Models

• URL: http://arxiv.org/abs/2504.08746v1
• Date: Mon, 24 Mar 2025 09:03:12 GMT
• Title: Enhancing Recommender Systems Using Textual Embeddings from Pre-trained Language Models
• Authors: Ngoc Luyen Le, Marie-Hélène Abel,
• Abstract summary: In this paper, we explore enhancing recommender systems using textual embeddings from pre-trained language models.<n>Our experiments demonstrate that this approach significantly improves recommendation accuracy and relevance, resulting in more personalized and context-aware recommendations.
• Score: 2.3020018305241337
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advancements in language models and pre-trained language models like BERT and RoBERTa have revolutionized natural language processing, enabling a deeper understanding of human-like language. In this paper, we explore enhancing recommender systems using textual embeddings from pre-trained language models to address the limitations of traditional recommender systems that rely solely on explicit features from users, items, and user-item interactions. By transforming structured data into natural language representations, we generate high-dimensional embeddings that capture deeper semantic relationships between users, items, and contexts. Our experiments demonstrate that this approach significantly improves recommendation accuracy and relevance, resulting in more personalized and context-aware recommendations. The findings underscore the potential of PLMs to enhance the effectiveness of recommender systems.

Related papers

• Align-SLM: Textless Spoken Language Models with Reinforcement Learning from AI Feedback [50.84142264245052]
  This work introduces the Align-SLM framework to enhance the semantic understanding of textless Spoken Language Models (SLMs) Our approach generates multiple speech continuations from a given prompt and uses semantic metrics to create preference data for Direct Preference Optimization (DPO) We evaluate the framework using ZeroSpeech 2021 benchmarks for lexical and syntactic modeling, the spoken version of the StoryCloze dataset for semantic coherence, and other speech generation metrics, including the GPT4-o score and human evaluation.
  arXiv  Detail & Related papers  (2024-11-04T06:07:53Z)
• Language Representations Can be What Recommenders Need: Findings and Potentials [57.90679739598295]
  We show that item representations, when linearly mapped from advanced LM representations, yield superior recommendation performance. This outcome suggests the possible homomorphism between the advanced language representation space and an effective item representation space for recommendation. Our findings highlight the connection between language modeling and behavior modeling, which can inspire both natural language processing and recommender system communities.
  arXiv  Detail & Related papers  (2024-07-07T17:05:24Z)
• Emerging Synergies Between Large Language Models and Machine Learning in Ecommerce Recommendations [19.405233437533713]
  Large language models (LLMs) have superior capabilities in basic tasks of language understanding and generation. We introduce a representative approach to learning user and item representations using LLM as a feature encoder. We then reviewed the latest advances in LLMs techniques for collaborative filtering enhanced recommendation systems.
  arXiv  Detail & Related papers  (2024-03-05T08:31:00Z)
• Exploring the Impact of Large Language Models on Recommender Systems: An Extensive Review [2.780460221321639]
  The paper underscores the significance of Large Language Models in reshaping recommender systems. LLMs exhibit exceptional proficiency in recommending items, showcasing their adeptness in comprehending intricacies of language. Despite their transformative potential, challenges persist, including sensitivity to input prompts, occasional misinterpretations, and unforeseen recommendations.
  arXiv  Detail & Related papers  (2024-02-11T00:24:17Z)
• User Embedding Model for Personalized Language Prompting [9.472634942498859]
  We introduce a new User Embedding Module (UEM) that efficiently processes user history in free-form text by compressing and representing them as embeddings. Our experiments demonstrate the superior capability of this approach in handling significantly longer histories. The main contribution of this research is to demonstrate the ability to bias language models with user signals represented as embeddings.
  arXiv  Detail & Related papers  (2024-01-10T00:35:52Z)
• Reformulating Sequential Recommendation: Learning Dynamic User Interest with Content-enriched Language Modeling [18.297332953450514]
  We propose LANCER, which leverages the semantic understanding capabilities of pre-trained language models to generate personalized recommendations. Our approach bridges the gap between language models and recommender systems, resulting in more human-like recommendations.
  arXiv  Detail & Related papers  (2023-09-19T08:54:47Z)
• Exploring Large Language Model for Graph Data Understanding in Online Job Recommendations [63.19448893196642]
  We present a novel framework that harnesses the rich contextual information and semantic representations provided by large language models to analyze behavior graphs. By leveraging this capability, our framework enables personalized and accurate job recommendations for individual users.
  arXiv  Detail & Related papers  (2023-07-10T11:29:41Z)
• Recommender Systems in the Era of Large Language Models (LLMs) [62.0129013439038]
  Large Language Models (LLMs) have revolutionized the fields of Natural Language Processing (NLP) and Artificial Intelligence (AI) We conduct a comprehensive review of LLM-empowered recommender systems from various aspects including Pre-training, Fine-tuning, and Prompting.
  arXiv  Detail & Related papers  (2023-07-05T06:03:40Z)
• Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approach [83.62750225073341]
  We consider recommendation as instruction following by large language models (LLMs) We first design a general instruction format for describing the preference, intention, task form and context of a user in natural language. Then we manually design 39 instruction templates and automatically generate a large amount of user-personalized instruction data.
  arXiv  Detail & Related papers  (2023-05-11T17:39:07Z)
• Chain of Hindsight Aligns Language Models with Feedback [62.68665658130472]
  We propose a novel technique, Chain of Hindsight, that is easy to optimize and can learn from any form of feedback, regardless of its polarity. We convert all types of feedback into sequences of sentences, which are then used to fine-tune the model. By doing so, the model is trained to generate outputs based on feedback, while learning to identify and correct negative attributes or errors.
  arXiv  Detail & Related papers  (2023-02-06T10:28:16Z)
• Finetuning Large-Scale Pre-trained Language Models for Conversational Recommendation with Knowledge Graph [35.033130888779226]
  We present a pre-trained language model (PLM) based framework called RID conversational recommender system (CRS) RID significantly outperforms the state-of-the-art methods on both evaluations of dialogue and recommendation.
  arXiv  Detail & Related papers  (2021-10-14T15:49:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.