ELCoRec: Enhance Language Understanding with Co-Propagation of Numerical and Categorical Features for Recommendation

• URL: http://arxiv.org/abs/2406.18825v1
• Date: Thu, 27 Jun 2024 01:37:57 GMT
• Title: ELCoRec: Enhance Language Understanding with Co-Propagation of Numerical and Categorical Features for Recommendation
• Authors: Jizheng Chen, Kounianhua Du, Jianghao Lin, Bo Chen, Ruiming Tang, Weinan Zhang,
• Abstract summary: Large language models have been flourishing in the natural language processing (NLP) domain. Despite the intelligence shown by the recommendation-oriented finetuned models, LLMs struggle to fully understand the user behavior patterns. Existing works only fine-tune a sole LLM on given text data without introducing that important information to it.
• Score: 38.64175351885443
• License:
• Abstract: Large language models have been flourishing in the natural language processing (NLP) domain, and their potential for recommendation has been paid much attention to. Despite the intelligence shown by the recommendation-oriented finetuned models, LLMs struggle to fully understand the user behavior patterns due to their innate weakness in interpreting numerical features and the overhead for long context, where the temporal relations among user behaviors, subtle quantitative signals among different ratings, and various side features of items are not well explored. Existing works only fine-tune a sole LLM on given text data without introducing that important information to it, leaving these problems unsolved. In this paper, we propose ELCoRec to Enhance Language understanding with CoPropagation of numerical and categorical features for Recommendation. Concretely, we propose to inject the preference understanding capability into LLM via a GAT expert model where the user preference is better encoded by parallelly propagating the temporal relations, and rating signals as well as various side information of historical items. The parallel propagation mechanism could stabilize heterogeneous features and offer an informative user preference encoding, which is then injected into the language models via soft prompting at the cost of a single token embedding. To further obtain the user's recent interests, we proposed a novel Recent interaction Augmented Prompt (RAP) template. Experiment results over three datasets against strong baselines validate the effectiveness of ELCoRec. The code is available at https://anonymous.4open.science/r/CIKM_Code_Repo-E6F5/README.md.

Related papers

• Beyond Retrieval: Generating Narratives in Conversational Recommender Systems [4.912663905306209]
  We introduce a new dataset (REGEN) for natural language generation tasks in conversational recommendations. We establish benchmarks using well-known generative metrics, and perform an automated evaluation of the new dataset using a rater LLM. And to the best of our knowledge, represents the first attempt to analyze the capabilities of LLMs in understanding recommender signals and generating rich narratives.
  arXiv  Detail & Related papers  (2024-10-22T07:53:41Z)
• Aligning Large Language Models with Self-generated Preference Data [72.99676237703099]
  We propose a new framework that boosts the alignment of large language models (LLMs) with human preferences. Our key idea is leveraging the human prior knowledge within the small (seed) data. We introduce a noise-aware preference learning algorithm to mitigate the risk of low quality within generated preference data.
  arXiv  Detail & Related papers  (2024-06-06T18:01:02Z)
• CLAIM Your Data: Enhancing Imputation Accuracy with Contextual Large Language Models [0.18416014644193068]
  This paper introduces the Contextual Language model for Accurate Imputation Method (CLAIM) Unlike traditional imputation methods, CLAIM utilizes contextually relevant natural language descriptors to fill missing values. Our evaluations across diverse datasets and missingness patterns reveal CLAIM's superior performance over existing imputation techniques.
  arXiv  Detail & Related papers  (2024-05-28T00:08:29Z)
• CELA: Cost-Efficient Language Model Alignment for CTR Prediction [71.85120354973073]
  Click-Through Rate (CTR) prediction holds a paramount position in recommender systems. Recent efforts have sought to mitigate these challenges by integrating Pre-trained Language Models (PLMs) We propose textbfCost-textbfEfficient textbfLanguage Model textbfAlignment (textbfCELA) for CTR prediction.
  arXiv  Detail & Related papers  (2024-05-17T07:43:25Z)
• RecExplainer: Aligning Large Language Models for Explaining Recommendation Models [50.74181089742969]
  Large language models (LLMs) have demonstrated remarkable intelligence in understanding, reasoning, and instruction following. This paper presents the initial exploration of using LLMs as surrogate models to explain black-box recommender models. To facilitate an effective alignment, we introduce three methods: behavior alignment, intention alignment, and hybrid alignment.
  arXiv  Detail & Related papers  (2023-11-18T03:05:43Z)
• Representation Learning with Large Language Models for Recommendation [34.46344639742642]
  We propose a model-agnostic framework RLMRec to enhance recommenders with large language models (LLMs)empowered representation learning. RLMRec incorporates auxiliary textual signals, develops a user/item profiling paradigm empowered by LLMs, and aligns the semantic space of LLMs with the representation space of collaborative relational signals.
  arXiv  Detail & Related papers  (2023-10-24T15:51:13Z)
• Rethinking the Evaluation for Conversational Recommendation in the Era of Large Language Models [115.7508325840751]
  The recent success of large language models (LLMs) has shown great potential to develop more powerful conversational recommender systems (CRSs) In this paper, we embark on an investigation into the utilization of ChatGPT for conversational recommendation, revealing the inadequacy of the existing evaluation protocol. We propose an interactive Evaluation approach based on LLMs named iEvaLM that harnesses LLM-based user simulators.
  arXiv  Detail & Related papers  (2023-05-22T15:12:43Z)
• Offline RL for Natural Language Generation with Implicit Language Q Learning [87.76695816348027]
  Large language models can be inconsistent when it comes to completing user specified tasks. We propose a novel RL method, that combines both the flexible utility framework of RL with the ability of supervised learning. In addition to empirically validating ILQL, we present a detailed empirical analysis situations where offline RL can be useful in natural language generation settings.
  arXiv  Detail & Related papers  (2022-06-05T18:38:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.