PreferRec: Learning and Transferring Pareto Preferences for Multi-objective Re-ranking

• URL: http://arxiv.org/abs/2603.22073v1
• Date: Mon, 23 Mar 2026 15:04:33 GMT
• Title: PreferRec: Learning and Transferring Pareto Preferences for Multi-objective Re-ranking
• Abstract summary: We propose PreferRec, a novel framework that explicitly models and transfers Pareto preferences across users.<n>Specifically, PreferRec aims to capture user intrinsic trade-offs among multiple conflicting objectives at the intent level.<n>Knowledge-Guided Transfer facilitates efficient cross-user knowledge transfer.
• Score: 28.35373733496984
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-objective re-ranking has become a critical component of modern multi-stage recommender systems, as it tasked to balance multiple conflicting objectives such as accuracy, diversity, and fairness. Existing multi-objective re-ranking methods typically optimize aggregate objectives at the item level using static or handcrafted preference weights. This design overlooks that users inherently exhibit Pareto-optimal preferences at the intent level, reflecting personalized trade-offs among objectives rather than fixed weight combinations. Moreover, most approaches treat re-ranking task for each user as an isolated problem, and repeatedly learn the preferences from scratch. Such a paradigm not only incurs high computational cost, but also ignores the fact that users often share similar preference trade-off structures across objectives. Inspired by the existence of homogeneous multi-objective optimization spaces where Pareto-optimal patterns are transferable, we propose PreferRec, a novel framework that explicitly models and transfers Pareto preferences across users. Specifically, PreferRec is built upon three tightly coupled components: Preference-Aware Pareto Learning aims to capture user intrinsic trade-offs among multiple conflicting objectives at the intent level. By learning Pareto preference representations from re-ranking populations, this component explicitly models how users prioritize different objectives under diverse contexts. Knowledge-Guided Transfer facilitates efficient cross-user knowledge transfer by distilling shared optimization patterns across homogeneous optimization spaces. The transferred knowledge is then used to guide solution selection and personalized re-ranking, biasing the optimization process toward high-quality regions of the Pareto front while preserving user-specific preference characteristics.

Related papers

• Efficient Controller Learning from Human Preferences and Numerical Data Via Multi-Modal Surrogate Models [1.2526963688768455]
  We propose a multi-fidelity, multi-modal Bayesian optimization framework that integrates low-fidelity numerical data with high-fidelity human preferences.<n>We illustrate the framework by tuning an autonomous vehicle's trajectory planner, showing that combining numerical and preference data significantly reduces the need for experiments involving the human decision maker.
  arXiv  Detail & Related papers  (2026-03-25T10:04:03Z)
• Synthetic Interaction Data for Scalable Personalization in Large Language Models [67.31884245564086]
  We introduce a high-fidelity synthetic data generation framework called PersonaGym.<n>Unlike prior work that treats personalization as static persona-preference pairs, PersonaGym models a dynamic preference process.<n>We release PersonaAtlas, a large-scale, high-quality, and diverse synthetic dataset of high-fidelity multi-turn personalized interaction trajectories.
  arXiv  Detail & Related papers  (2026-02-12T20:41:22Z)
• Tree of Preferences for Diversified Recommendation [54.183647833064136]
  We study diversified recommendation from a data-bias perspective.<n>Inspired by the outstanding performance of large language models (LLMs) in zero-shot inference leveraging world knowledge, we propose a novel approach.
  arXiv  Detail & Related papers  (2025-12-24T04:13:17Z)
• LoRe: Personalizing LLMs via Low-Rank Reward Modeling [47.12507639759984]
  We introduce a novel framework that leverages low-rank preference modeling to efficiently learn and generalize user-specific reward functions.<n>We validate our method on multiple preference datasets, demonstrating superior generalization to unseen users and improved accuracy in preference prediction tasks.
  arXiv  Detail & Related papers  (2025-04-20T01:16:24Z)
• Preference-Guided Diffusion for Multi-Objective Offline Optimization [64.08326521234228]
  We propose a preference-guided diffusion model for offline multi-objective optimization.<n>Our guidance is a preference model trained to predict the probability that one design dominates another.<n>Our results highlight the effectiveness of classifier-guided diffusion models in generating diverse and high-quality solutions.
  arXiv  Detail & Related papers  (2025-03-21T16:49:38Z)
• Provably Efficient Multi-Objective Bandit Algorithms under Preference-Centric Customization [24.533662423325943]
  We study a preference-aware MO-MAB framework in the presence of explicit user preference.<n>This is the first theoretical study of customized MO-MAB optimization with explicit user preferences.
  arXiv  Detail & Related papers  (2025-02-19T06:06:13Z)
• Uncertain Multi-Objective Recommendation via Orthogonal Meta-Learning Enhanced Bayesian Optimization [30.031396809114625]
  We introduce a novel framework that categorizes RS autonomy into five distinct levels, ranging from basic rule-based accuracy-driven systems to behavior-aware, uncertain multi-objective RSs.<n>We propose an approach that dynamically identifies and optimize multiple objectives based on individual user preferences, fostering more ethical and intelligent user-centric recommendations.
  arXiv  Detail & Related papers  (2025-02-18T08:10:09Z)
• Few-shot Steerable Alignment: Adapting Rewards and LLM Policies with Neural Processes [50.544186914115045]
  Large language models (LLMs) are increasingly embedded in everyday applications.<n> Ensuring their alignment with the diverse preferences of individual users has become a critical challenge.<n>We present a novel framework for few-shot steerable alignment.
  arXiv  Detail & Related papers  (2024-12-18T16:14:59Z)
• Comparison-based Active Preference Learning for Multi-dimensional Personalization [7.349038301460469]
  Large language models (LLMs) have shown remarkable success, but aligning them with human preferences remains a core challenge.<n>Recent studies have explored multi-dimensional personalization, which aims to enable models to generate responses personalized to explicit preferences.<n>We propose Active Multi-dimensional Preference Learning (AMPLe), designed to capture implicit user preferences from interactively collected comparative feedback.
  arXiv  Detail & Related papers  (2024-11-01T11:49:33Z)
• Unlocking Decoding-time Controllability: Gradient-Free Multi-Objective Alignment with Contrastive Prompts [38.95012734839997]
  Multi-objective alignment aims at balancing and controlling the different alignment objectives of large language models. We propose MCA (Multi-objective Contrastive Alignemnt), which constructs an expert prompt and an adversarial prompt for each objective to contrast.
  arXiv  Detail & Related papers  (2024-08-09T14:36:42Z)
• Deep Pareto Reinforcement Learning for Multi-Objective Recommender Systems [54.2484458418885]
  optimizing multiple objectives simultaneously is an important task for recommendation platforms.<n>Existing multi-objective recommender systems do not systematically consider such dynamic relationships.
  arXiv  Detail & Related papers  (2024-07-04T02:19:49Z)
• Controllable Preference Optimization: Toward Controllable Multi-Objective Alignment [103.12563033438715]
  Alignment in artificial intelligence pursues consistency between model responses and human preferences as well as values. Existing alignment techniques are mostly unidirectional, leading to suboptimal trade-offs and poor flexibility over various objectives. We introduce controllable preference optimization (CPO), which explicitly specifies preference scores for different objectives.
  arXiv  Detail & Related papers  (2024-02-29T12:12:30Z)
• Beyond One-Preference-Fits-All Alignment: Multi-Objective Direct Preference Optimization [76.09576643028362]
  We present Multi-Objective Direct Preference Optimization (MODPO) for multiple alignment objectives. MODPO folds language modeling directly into reward modeling, training language models as implicit collective reward models. It theoretically yields the same optimal solutions as MORLHF but is practically more stable and efficient.
  arXiv  Detail & Related papers  (2023-10-05T17:35:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.