Unifying Search and Recommendation in LLMs via Gradient Multi-Subspace Tuning

• URL: http://arxiv.org/abs/2601.09496v1
• Date: Wed, 14 Jan 2026 14:03:07 GMT
• Title: Unifying Search and Recommendation in LLMs via Gradient Multi-Subspace Tuning
• Authors: Jujia Zhao, Zihan Wang, Shuaiqun Pan, Suzan Verberne, Zhaochun Ren,
• Abstract summary: Gradient Multi-Subspace Tuning (GEMS) is a novel framework that unifies search and recommendation tasks.<n>We show that GEMS consistently outperforms the state-of-the-art baselines across both search and recommendation tasks.
• Score: 33.69176756907003
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Search and recommendation (S&R) are core to online platforms, addressing explicit intent through queries and modeling implicit intent from behaviors, respectively. Their complementary roles motivate a unified modeling paradigm. Early studies to unify S&R adopt shared encoders with task-specific heads, while recent efforts reframe item ranking in both S&R as conditional generation. The latter holds particular promise, enabling end-to-end optimization and leveraging the semantic understanding of LLMs. However, existing methods rely on full fine-tuning, which is computationally expensive and limits scalability. Parameter-efficient fine-tuning (PEFT) offers a more practical alternative but faces two critical challenges in unifying S&R: (1) gradient conflicts across tasks due to divergent optimization objectives, and (2) shifts in user intent understanding caused by overfitting to fine-tuning data, which distort general-domain knowledge and weaken LLM reasoning. To address the above issues, we propose Gradient Multi-Subspace Tuning (GEMS), a novel framework that unifies S&R with LLMs while alleviating gradient conflicts and preserving general-domain knowledge. GEMS introduces (1) \textbf{Multi-Subspace Decomposition}, which disentangles shared and task-specific optimization signals into complementary low-rank subspaces, thereby reducing destructive gradient interference, and (2) \textbf{Null-Space Projection}, which constrains parameter updates to a subspace orthogonal to the general-domain knowledge space, mitigating shifts in user intent understanding. Extensive experiments on benchmark datasets show that GEMS consistently outperforms the state-of-the-art baselines across both search and recommendation tasks, achieving superior effectiveness.

Related papers

• OmniVL-Guard: Towards Unified Vision-Language Forgery Detection and Grounding via Balanced RL [63.388513841293616]
  Existing forgery detection methods fail to handle the interleaved text, images, and videos prevalent in real-world misinformation.<n>To bridge this gap, this paper targets to develop a unified framework for omnibus vision-language forgery detection and grounding.<n>We propose textbf OmniVL-Guard, a balanced reinforcement learning framework for omnibus vision-language forgery detection and grounding.
  arXiv  Detail & Related papers  (2026-02-11T09:41:36Z)
• R2LED: Equipping Retrieval and Refinement in Lifelong User Modeling with Semantic IDs for CTR Prediction [23.668401664583758]
  We propose a novel paradigm that equips retrieval and refinement in Lifelong User Modeling with SEmantic IDs (R2LED)<n>First, we introduce a Multi-route Mixed Retrieval for the retrieval stage. On the other hand, a mixed retrieval mechanism is proposed to efficiently retrieve candidates from both collaborative and semantic views.<n>For refinement, we design a Bi-level Fusion Refinement, including a target-aware cross-attention for route-level fusion and a gate mechanism for SID-level fusion.
  arXiv  Detail & Related papers  (2026-02-06T11:27:20Z)
• LLM-EDT: Large Language Model Enhanced Cross-domain Sequential Recommendation with Dual-phase Training [53.539682966282534]
  Cross-domain Sequential Recommendation (CDSR) has been proposed to enrich user-item interactions by incorporating information from various domains.<n>Despite current progress, the imbalance issue and transition issue hinder further development of CDSR.<n>We propose an LLMs Enhanced Cross-domain Sequential Recommendation with Dual-phase Training (LLM-EDT)
  arXiv  Detail & Related papers  (2025-11-25T05:18:04Z)
• UR$^2$: Unify RAG and Reasoning through Reinforcement Learning [17.319590573147565]
  Large Language Models (LLMs) have shown remarkable capabilities through two complementary paradigms: Retrieval-Augmented Generation (RAG) and Reinforcement Learning from Verifiable Rewards (RLVR)<n>We propose UR2 (Unified RAG and Reasoning), a general framework that unifies retrieval and reasoning through reinforcement learning.<n>Experiments across open-domain QA, MMLU-Pro, medical, and mathematical reasoning tasks demonstrate that UR$2$ significantly outperforms existing RAG and RL methods.
  arXiv  Detail & Related papers  (2025-08-08T09:33:20Z)
• NDCG-Consistent Softmax Approximation with Accelerated Convergence [67.10365329542365]
  We propose novel loss formulations that align directly with ranking metrics.<n>We integrate the proposed RG losses with the highly efficient Alternating Least Squares (ALS) optimization method.<n> Empirical evaluations on real-world datasets demonstrate that our approach achieves comparable or superior ranking performance.
  arXiv  Detail & Related papers  (2025-06-11T06:59:17Z)
• Exploiting Edited Large Language Models as General Scientific Optimizers [10.489095729605152]
  Large language models (LLMs) have been widely adopted in mathematical optimization in scientific scenarios.<n>We propose a conceptually simple and general bi-level optimization method, namely textbfGeneral textbfScientific textbfOptimizers (GSO)
  arXiv  Detail & Related papers  (2025-03-08T18:01:11Z)
• How to Alleviate Catastrophic Forgetting in LLMs Finetuning? Hierarchical Layer-Wise and Element-Wise Regularization [15.434072331989878]
  Large Language Models (LLMs) exhibit strong general language capabilities.<n>Fine-tuning these models on domain-specific tasks often leads to catastrophic forgetting, where the model overwrites or loses essential knowledge acquired during pretraining.<n>We propose a novel approach to compute the element-wise importance of model parameters crucial for preserving general knowledge during fine-tuning.
  arXiv  Detail & Related papers  (2025-01-23T13:54:53Z)
• LLMEmb: Large Language Model Can Be a Good Embedding Generator for Sequential Recommendation [57.49045064294086]
  Large Language Model (LLM) has the ability to capture semantic relationships between items, independent of their popularity.<n>We introduce LLMEmb, a novel method leveraging LLM to generate item embeddings that enhance Sequential Recommender Systems (SRS) performance.
  arXiv  Detail & Related papers  (2024-09-30T03:59:06Z)
• SHERL: Synthesizing High Accuracy and Efficient Memory for Resource-Limited Transfer Learning [63.93193829913252]
  We propose an innovative METL strategy called SHERL for resource-limited scenarios. In the early route, intermediate outputs are consolidated via an anti-redundancy operation. In the late route, utilizing minimal late pre-trained layers could alleviate the peak demand on memory overhead.
  arXiv  Detail & Related papers  (2024-07-10T10:22:35Z)
• Robust Analysis of Multi-Task Learning Efficiency: New Benchmarks on Light-Weighed Backbones and Effective Measurement of Multi-Task Learning Challenges by Feature Disentanglement [69.51496713076253]
  In this paper, we focus on the aforementioned efficiency aspects of existing MTL methods. We first carry out large-scale experiments of the methods with smaller backbones and on a the MetaGraspNet dataset as a new test ground. We also propose Feature Disentanglement measure as a novel and efficient identifier of the challenges in MTL.
  arXiv  Detail & Related papers  (2024-02-05T22:15:55Z)
• Learning towards Synchronous Network Memorizability and Generalizability for Continual Segmentation across Multiple Sites [52.84959869494459]
  In clinical practice, a segmentation network is often required to continually learn on a sequential data stream from multiple sites. Existing methods are usually restricted in either network memorizability on previous sites or generalizability on unseen sites. This paper aims to tackle the problem of Synchronous Memorizability and Generalizability with a novel proposed SMG-learning framework.
  arXiv  Detail & Related papers  (2022-06-14T13:04:36Z)
• Semi-supervised Domain Adaptive Structure Learning [72.01544419893628]
  Semi-supervised domain adaptation (SSDA) is a challenging problem requiring methods to overcome both 1) overfitting towards poorly annotated data and 2) distribution shift across domains. We introduce an adaptive structure learning method to regularize the cooperation of SSL and DA.
  arXiv  Detail & Related papers  (2021-12-12T06:11:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.