Selective LLM-Guided Regularization for Enhancing Recommendation Models

• URL: http://arxiv.org/abs/2512.21526v1
• Date: Thu, 25 Dec 2025 06:30:00 GMT
• Title: Selective LLM-Guided Regularization for Enhancing Recommendation Models
• Abstract summary: We introduce a model-agnostic and efficient framework that activates LLM based pairwise ranking supervision only when a trainable gating mechanism informs by user history length, item popularity, and model uncertainty predicts the LLM to be reliable.<n> Experiments across multiple datasets show that this selective strategy consistently improves overall accuracy and yields substantial gains in cold start and long tail regimes, outperforming global distillation baselines.
• Score: 7.406718588794206
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models provide rich semantic priors and strong reasoning capabilities, making them promising auxiliary signals for recommendation. However, prevailing approaches either deploy LLMs as standalone recommender or apply global knowledge distillation, both of which suffer from inherent drawbacks. Standalone LLM recommender are costly, biased, and unreliable across large regions of the user item space, while global distillation forces the downstream model to imitate LLM predictions even when such guidance is inaccurate. Meanwhile, recent studies show that LLMs excel particularly in re-ranking and challenging scenarios, rather than uniformly across all contexts.We introduce Selective LLM Guided Regularization, a model-agnostic and computation efficient framework that activates LLM based pairwise ranking supervision only when a trainable gating mechanism informing by user history length, item popularity, and model uncertainty predicts the LLM to be reliable. All LLM scoring is performed offline, transferring knowledge without increasing inference cost. Experiments across multiple datasets show that this selective strategy consistently improves overall accuracy and yields substantial gains in cold start and long tail regimes, outperforming global distillation baselines.

Related papers

• De-attribute to Forget for LLM Unlearning [79.82535763351594]
  The rapid development of large language models (LLMs) has raised concerns on the use of inappropriate data for training.<n>This paper proposes the first LLM unlearning framework based on data attribution rewards called DareU.
  arXiv  Detail & Related papers  (2026-05-29T07:03:20Z)
• Memory-Based Advantage Shaping for LLM-Guided Reinforcement Learning [18.215893951726166]
  In environments with sparse or delayed rewards, reinforcement learning incurs high sample complexity.<n>This limitation has motivated the use of large language models (LLMs) for subgoal discovery and trajectory guidance.<n>We address these challenges by constructing a memory graph that encodes subgoals and trajectories from both LLM guidance and the agent's own successful rollouts.
  arXiv  Detail & Related papers  (2026-02-20T01:44:35Z)
• Rethinking the Role of LLMs in Time Series Forecasting [15.951870420397682]
  Large language models (LLMs) have been introduced to time series forecasting (TSF) to incorporate contextual knowledge beyond numerical signals.<n>We show that such conclusions stem from limited evaluation settings and do not hold at scale.<n>Our results demonstrate that emphLLM4TS indeed improves forecasting performance, with especially large gains in cross-domain generalization.
  arXiv  Detail & Related papers  (2026-02-16T13:39:09Z)
• LLM as Explainable Re-Ranker for Recommendation System [3.0720618129954875]
  Traditional recommendation systems often lack explainability and suffer from issues such as popularity bias.<n>We propose to use large language models (LLMs) as an explainable re-ranker to enhance both accuracy and interpretability.
  arXiv  Detail & Related papers  (2025-12-03T04:42:58Z)
• Reasoning with Confidence: Efficient Verification of LLM Reasoning Steps via Uncertainty Heads [104.9566359759396]
  We propose a lightweight alternative for step-level reasoning verification based on data-driven uncertainty scores.<n>Our findings suggest that the internal states of LLMs encode their uncertainty and can serve as reliable signals for reasoning verification.
  arXiv  Detail & Related papers  (2025-11-09T03:38:29Z)
• LLM-Lasso: A Robust Framework for Domain-Informed Feature Selection and Regularization [59.75242204923353]
  We introduce LLM-Lasso, a framework that leverages large language models (LLMs) to guide feature selection in Lasso regression.<n>LLMs generate penalty factors for each feature, which are converted into weights for the Lasso penalty using a simple, tunable model.<n>Features identified as more relevant by the LLM receive lower penalties, increasing their likelihood of being retained in the final model.
  arXiv  Detail & Related papers  (2025-02-15T02:55:22Z)
• A Little Help Goes a Long Way: Efficient LLM Training by Leveraging Small LMs [74.35290684163718]
  A primary challenge in large language model (LLM) development is their onerous pre-training cost. This paper explores a promising paradigm to improve LLM pre-training efficiency and quality by leveraging a small language model (SLM)
  arXiv  Detail & Related papers  (2024-10-24T14:31:52Z)
• One Token Can Help! Learning Scalable and Pluggable Virtual Tokens for Retrieval-Augmented Large Language Models [67.49462724595445]
  Retrieval-augmented generation (RAG) is a promising way to improve large language models (LLMs)<n>We propose a novel method that involves learning scalable and pluggable virtual tokens for RAG.
  arXiv  Detail & Related papers  (2024-05-30T03:44:54Z)
• Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models [52.98743860365194]
  We propose a new fine-tuning method called Self-Play fIne-tuNing (SPIN) At the heart of SPIN lies a self-play mechanism, where the LLM refines its capability by playing against instances of itself. This sheds light on the promise of self-play, enabling the achievement of human-level performance in LLMs without the need for expert opponents.
  arXiv  Detail & Related papers  (2024-01-02T18:53:13Z)
• On Learning to Summarize with Large Language Models as References [101.79795027550959]
  Large language models (LLMs) are favored by human annotators over the original reference summaries in commonly used summarization datasets. We study an LLM-as-reference learning setting for smaller text summarization models to investigate whether their performance can be substantially improved.
  arXiv  Detail & Related papers  (2023-05-23T16:56:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.