Reason4Rec: Large Language Models for Recommendation with Deliberative User Preference Alignment

• URL: http://arxiv.org/abs/2502.02061v2
• Date: Mon, 17 Feb 2025 11:22:58 GMT
• Title: Reason4Rec: Large Language Models for Recommendation with Deliberative User Preference Alignment
• Authors: Yi Fang, Wenjie Wang, Yang Zhang, Fengbin Zhu, Qifan Wang, Fuli Feng, Xiangnan He,
• Abstract summary: We propose a new Deliberative Recommendation task, which incorporates explicit reasoning about user preferences as an additional alignment goal.<n>We then introduce the Reasoning-powered Recommender framework for deliberative user preference alignment.
• Score: 69.11529841118671
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While recent advancements in aligning Large Language Models (LLMs) with recommendation tasks have shown great potential and promising performance overall, these aligned recommendation LLMs still face challenges in complex scenarios. This is primarily due to the current alignment approach focusing on optimizing LLMs to generate user feedback directly, without incorporating deliberation. To overcome this limitation and develop more reliable LLMs for recommendations, we propose a new Deliberative Recommendation task, which incorporates explicit reasoning about user preferences as an additional alignment goal. We then introduce the Reasoning-powered Recommender framework for deliberative user preference alignment, designed to enhance reasoning capabilities by utilizing verbalized user feedback in a step-wise manner to tackle this task. The framework employs collaborative step-wise experts and tailored training strategies for each expert. Experimental results across three real-world datasets demonstrate the rationality of the deliberative task formulation and the superior performance of the proposed framework in improving both prediction accuracy and reasoning quality.

Related papers

• Large Language Model Empowered Recommendation Meets All-domain Continual Pre-Training [60.38082979765664]
  CPRec is an All-domain Continual Pre-Training framework for Recommendation. It holistically align LLMs with universal user behaviors through the continual pre-training paradigm. We conduct experiments on five real-world datasets from two distinct platforms.
  arXiv  Detail & Related papers  (2025-04-11T20:01:25Z)
• Self-Improvement Towards Pareto Optimality: Mitigating Preference Conflicts in Multi-Objective Alignment [74.25832963097658]
  Multi-Objective Alignment (MOA) aims to align responses with multiple human preference objectives. We find that DPO-based MOA approaches suffer from widespread preference conflicts in the data.
  arXiv  Detail & Related papers  (2025-02-20T08:27:00Z)
• Semantic Convergence: Harmonizing Recommender Systems via Two-Stage Alignment and Behavioral Semantic Tokenization [10.47505806629852]
  Large language models (LLMs) are adept at discerning profound user interests from historical behaviors.<n>We propose a novel framework that harmoniously merges traditional recommendation models with the prowess of LLMs.<n>We design a series of specialized supervised learning tasks aimed at aligning collaborative signals with the subtleties of natural language semantics.
  arXiv  Detail & Related papers  (2024-12-18T12:07:58Z)
• Dynamic Rewarding with Prompt Optimization Enables Tuning-free Self-Alignment of Language Models [54.381650481255235]
  We introduce a new tuning-free approach for self-alignment, Dynamic Rewarding with Prompt Optimization (O) Our approach leverages a search-based optimization framework that allows LLMs to iteratively self-improve and craft the optimal alignment instructions. Empirical evaluations on eight recent LLMs, both open and closed-sourced, demonstrate that DRPO significantly enhances alignment performance.
  arXiv  Detail & Related papers  (2024-11-13T16:15:38Z)
• RosePO: Aligning LLM-based Recommenders with Human Values [38.029251417802044]
  We propose a general framework -- Recommendation with smoothing personalized Preference Optimization (RosePO) RosePO better aligns with customized human values during the post-training stage. Evaluation on three real-world datasets demonstrates the effectiveness of our method.
  arXiv  Detail & Related papers  (2024-10-16T12:54:34Z)
• RLRF4Rec: Reinforcement Learning from Recsys Feedback for Enhanced Recommendation Reranking [33.54698201942643]
  Large Language Models (LLMs) have demonstrated remarkable performance across diverse domains. This paper introduces RLRF4Rec, a novel framework integrating Reinforcement Learning from Recsys Feedback for Enhanced Recommendation Reranking.
  arXiv  Detail & Related papers  (2024-10-08T11:42:37Z)
• DaRec: A Disentangled Alignment Framework for Large Language Model and Recommender System [83.34921966305804]
  Large language models (LLMs) have demonstrated remarkable performance in recommender systems.<n>We propose a novel plug-and-play alignment framework for LLMs and collaborative models.<n>Our method is superior to existing state-of-the-art algorithms.
  arXiv  Detail & Related papers  (2024-08-15T15:56:23Z)
• LANE: Logic Alignment of Non-tuning Large Language Models and Online Recommendation Systems for Explainable Reason Generation [15.972926854420619]
  Leveraging large language models (LLMs) offers new opportunities for comprehensive recommendation logic generation. Fine-tuning LLM models for recommendation tasks incurs high computational costs and alignment issues with existing systems. In this work, our proposed effective strategy LANE aligns LLMs with online recommendation systems without additional LLMs tuning.
  arXiv  Detail & Related papers  (2024-07-03T06:20:31Z)
• Fairer Preferences Elicit Improved Human-Aligned Large Language Model Judgments [41.25558612970942]
  We show that large language models (LLMs) exhibit preference biases and worrying sensitivity to prompt designs. Motivated by this phenomenon, we propose an automatic Zero-shot Evaluation-oriented Prompt Optimization framework, ZEPO.
  arXiv  Detail & Related papers  (2024-06-17T09:48:53Z)
• Large Language Models are Learnable Planners for Long-Term Recommendation [59.167795967630305]
  Planning for both immediate and long-term benefits becomes increasingly important in recommendation. Existing methods apply Reinforcement Learning to learn planning capacity by maximizing cumulative reward for long-term recommendation. We propose to leverage the remarkable planning capabilities over sparse data of Large Language Models for long-term recommendation.
  arXiv  Detail & Related papers  (2024-02-29T13:49:56Z)
• Empowering Few-Shot Recommender Systems with Large Language Models -- Enhanced Representations [0.0]
  Large language models (LLMs) offer novel insights into tackling the few-shot scenarios encountered by explicit feedback-based recommender systems. Our study can inspire researchers to delve deeper into the multifaceted dimensions of LLMs's involvement in recommender systems.
  arXiv  Detail & Related papers  (2023-12-21T03:50:09Z)
• Choosing the Best of Both Worlds: Diverse and Novel Recommendations through Multi-Objective Reinforcement Learning [68.45370492516531]
  We introduce Scalarized Multi-Objective Reinforcement Learning (SMORL) for the Recommender Systems (RS) setting. SMORL agent augments standard recommendation models with additional RL layers that enforce it to simultaneously satisfy three principal objectives: accuracy, diversity, and novelty of recommendations. Our experimental results on two real-world datasets reveal a substantial increase in aggregate diversity, a moderate increase in accuracy, reduced repetitiveness of recommendations, and demonstrate the importance of reinforcing diversity and novelty as complementary objectives.
  arXiv  Detail & Related papers  (2021-10-28T13:22:45Z)
• Reward Constrained Interactive Recommendation with Natural Language Feedback [158.8095688415973]
  We propose a novel constraint-augmented reinforcement learning (RL) framework to efficiently incorporate user preferences over time. Specifically, we leverage a discriminator to detect recommendations violating user historical preference. Our proposed framework is general and is further extended to the task of constrained text generation.
  arXiv  Detail & Related papers  (2020-05-04T16:23:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.