RecRM-Bench: Benchmarking Multidimensional Reward Modeling for Agentic Recommender Systems

• URL: http://arxiv.org/abs/2605.11874v1
• Date: Tue, 12 May 2026 09:52:21 GMT
• Title: RecRM-Bench: Benchmarking Multidimensional Reward Modeling for Agentic Recommender Systems
• Abstract summary: We introduce RecRM-Bench, the largest and most comprehensive benchmark to date for agentic recommender systems.<n>It comprises over 1 million structured entries across four core evaluation dimensions: instruction following, factual consistency, query-item relevance, and fine-grained user behavior prediction.<n>We propose a systematic framework for the construction of multi-dimensional reward models and the integration of a hybrid reward function.
• Score: 40.152754832576996
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The integration of Large Language Model (LLM) agents is transforming recommender systems from simple query-item matching towards deeply personalized and interactive recommendations. Reinforcement Learning (RL) provides an essential framework for the optimization of these agents in recommendation tasks. However, current methodologies remain limited by a reliance on single dimensional outcome-based rewards that focus exclusively on final user interactions, overlooking critical intermediate capabilities, such as instruction following and complex intent understanding. Despite the necessity for designing multi-dimensional reward, the field lacks a standardized benchmark to facilitate this development. To bridge this gap, we introduce RecRM-Bench, the largest and most comprehensive benchmark to date for agentic recommender systems. It comprises over 1 million structured entries across four core evaluation dimensions: instruction following, factual consistency, query-item relevance, and fine-grained user behavior prediction. By supporting comprehensive assessment from syntactic compliance to complex intent grounding and preference modeling, RecRM-Bench provides a foundational dataset for training sophisticated reward models. Furthermore, we propose a systematic framework for the construction of multi-dimensional reward models and the integration of a hybrid reward function, establishing a robust foundation for developing reliable and highly capable agentic recommender systems. The complete RecRM-Bench dataset is publicly available at https://huggingface.co/datasets/wwzeng/RecRM-Bench.

Related papers

• Replacing Parameters with Preferences: Federated Alignment of Heterogeneous Vision-Language Models [63.70401095689976]
  Vision-Language Models (VLMs) have broad potential in privacy-sensitive domains such as healthcare and finance.<n>We propose MoR, a federated alignment framework that combines GRPO with Mixture-of-Rewards for heterogeneousVLMs.<n>MoR consistently outperforms federated alignment baselines in generalization and cross-client adaptability.
  arXiv  Detail & Related papers  (2026-05-05T07:02:50Z)
• Can Recommender Systems Teach Themselves? A Recursive Self-Improving Framework with Fidelity Control [82.30868101940068]
  We propose a paradigm in which a model bootstraps its own performance without reliance on external data or teacher models.<n>Our theoretical analysis shows that RSIR acts as a data-driven implicit regularizer, smoothing the optimization landscape.<n>We show that even smaller models benefit, and weak models can generate effective training curricula for stronger ones.
  arXiv  Detail & Related papers  (2026-02-17T15:31:32Z)
• SoliReward: Mitigating Susceptibility to Reward Hacking and Annotation Noise in Video Generation Reward Models [53.19726629537694]
  Post-training alignment of video generation models with human preferences is a critical goal.<n>Current data collection paradigms, reliant on in-prompt pairwise annotations, suffer from labeling noise.<n>We propose SoliReward, a systematic framework for video RM training.
  arXiv  Detail & Related papers  (2025-12-17T14:28:23Z)
• Multi-Agent Collaborative Reward Design for Enhancing Reasoning in Reinforcement Learning [13.30869366778628]
  CRM is a framework that replaces a single black-box reward model with a coordinated team of specialist evaluators.<n>To support training and assessment, we introduce rewardBench, a benchmark and training suite aligned with the collaborative structure of CRM.
  arXiv  Detail & Related papers  (2025-11-20T10:12:34Z)
• Revisiting scalable sequential recommendation with Multi-Embedding Approach and Mixture-of-Experts [15.976682531132676]
  We propose Fuxi-MME, a framework that integrates a multi-embedding strategy with a Mixture-of-Experts (MoE) architecture.<n>Specifically, to efficiently capture diverse item characteristics in a decoupled manner, we decompose the conventional single embedding matrix into several lower-dimensional embedding matrices.
  arXiv  Detail & Related papers  (2025-10-29T08:42:15Z)
• BaseReward: A Strong Baseline for Multimodal Reward Model [34.80724853211918]
  We introduce textbfBaseReward, a powerful and efficient baseline for multimodal reward modeling.<n>Our results show that BaseReward establishes a new SOTA on major benchmarks such as MM-RLHF-Reward Bench, VL-Reward Bench, and Multimodal Reward Bench.<n>To validate its practical utility beyond static benchmarks, we integrate BaseReward into a real-world reinforcement learning pipeline.
  arXiv  Detail & Related papers  (2025-09-19T16:25:26Z)
• STARec: An Efficient Agent Framework for Recommender Systems via Autonomous Deliberate Reasoning [54.28691219536054]
  We introduce STARec, a slow-thinking augmented agent framework that endows recommender systems with autonomous deliberative reasoning capabilities.<n>We develop anchored reinforcement training - a two-stage paradigm combining structured knowledge distillation from advanced reasoning models with preference-aligned reward shaping.<n>Experiments on MovieLens 1M and Amazon CDs benchmarks demonstrate that STARec achieves substantial performance gains compared with state-of-the-art baselines.
  arXiv  Detail & Related papers  (2025-08-26T08:47:58Z)
• SCoRE: Streamlined Corpus-based Relation Extraction using Multi-Label Contrastive Learning and Bayesian kNN [0.2812395851874055]
  We introduce SCoRE, a modular and cost-effective sentence-level relation extraction system.<n>SCoRE enables easy PLM switching, requires no finetuning, and adapts smoothly to diverse corpora and KGs.<n>We show that SCoRE matches or surpasses state-of-the-art methods while significantly reducing energy consumption.
  arXiv  Detail & Related papers  (2025-07-09T14:33:07Z)
• AgentRecBench: Benchmarking LLM Agent-based Personalized Recommender Systems [17.329692234349768]
  Agentic recommender systems are powered by Large Language Models (LLMs)<n>LLMs' advanced reasoning and role-playing capabilities enable autonomous, adaptive decision-making.<n>The field currently lacks standardized evaluation protocols to assess these methods.
  arXiv  Detail & Related papers  (2025-05-26T07:45:11Z)
• LLM4Rerank: LLM-based Auto-Reranking Framework for Recommendations [51.76373105981212]
  Reranking is a critical component in recommender systems, playing an essential role in refining the output of recommendation algorithms.<n>We introduce a comprehensive reranking framework, designed to seamlessly integrate various reranking criteria.<n>A customizable input mechanism is also integrated, enabling the tuning of the language model's focus to meet specific reranking needs.
  arXiv  Detail & Related papers  (2024-06-18T09:29:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.