MMGRid: Navigating Temporal-aware and Cross-domain Generative Recommendation via Model Merging

• URL: http://arxiv.org/abs/2601.15930v2
• Date: Thu, 29 Jan 2026 05:07:30 GMT
• Title: MMGRid: Navigating Temporal-aware and Cross-domain Generative Recommendation via Model Merging
• Authors: Tianjun Wei, Enneng Yang, Yingpeng Du, Huizhong Guo, Jie Zhang, Zhu Sun,
• Abstract summary: Generative Recommendation (GR) has emerged as a new paradigm in recommender systems (RSs)<n>We focus on a fundamental yet underexplored challenge in real-world: how to merge generative recommenders specialized to different real-world contexts.<n>We propose a unified framework MMGRid, a structured contextual grid of GR checkpoints that organizes models trained under diverse contexts.
• Score: 22.681048070167765
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Model merging (MM) offers an efficient mechanism for integrating multiple specialized models without access to original training data or costly retraining. While MM has demonstrated success in domains like computer vision, its role in recommender systems (RSs) remains largely unexplored. Recently, Generative Recommendation (GR) has emerged as a new paradigm in RSs, characterized by rapidly growing model scales and substantial computational costs, making MM particularly appealing for cost-sensitive deployment scenarios. In this work, we present the first systematic study of MM in GR through a contextual lens. We focus on a fundamental yet underexplored challenge in real-world: how to merge generative recommenders specialized to different real-world contexts, arising from temporal evolving user behaviors and heterogeneous application domains. To this end, we propose a unified framework MMGRid, a structured contextual grid of GR checkpoints that organizes models trained under diverse contexts induced by temporal evolution and domain diversity. All checkpoints are derived from a shared base LLM but fine-tuned on context-specific data, forming a realistic and controlled model space for systematically analyzing MM across GR paradigms and merging algorithms. Our investigation reveals several key insights. First, training GR models from LLMs can introduce parameter conflicts during merging due to token distribution shifts and objective disparities; such conflicts can be alleviated by disentangling task-aware and context-specific parameter changes via base model replacement. Second, incremental training across contexts induces recency bias, which can be effectively balanced through weighted contextual merging. Notably, we observe that optimal merging weights correlate with context-dependent interaction characteristics, offering practical guidance for weight selection in real-world deployments.

Related papers

• Co-GRPO: Co-Optimized Group Relative Policy Optimization for Masked Diffusion Model [74.99242687133408]
  Masked Diffusion Models (MDMs) have shown promising potential across vision, language, and cross-modal generation.<n>We introduce Co-GRPO, which reformulates MDM generation as a unified Markov Decision Process (MDP) that jointly incorporates both the model and the inference schedule.
  arXiv  Detail & Related papers  (2025-12-25T12:06:04Z)
• Large Language Models as Discounted Bayesian Filters [14.164508061248775]
  We introduce a Bayesian filtering framework to evaluate online inference in Large Language Models (LLMs)<n>We find that while LLM belief updates resemble Bayesian posteriors, they are more accurately characterized by an exponential forgetting filter with a model-specific discount factor smaller than one.<n>Although inherent priors are often miscalibrated, the updating mechanism itself remains structured and principled.
  arXiv  Detail & Related papers  (2025-12-20T19:56:39Z)
• RECALL: REpresentation-aligned Catastrophic-forgetting ALLeviation via Hierarchical Model Merging [33.22889542330089]
  Internal representations in large language models (LLMs) serve as reliable proxies of learned knowledge.<n>We propose RECALL, a representation-aware model merging framework for continual learning without access to historical data.
  arXiv  Detail & Related papers  (2025-10-23T12:17:37Z)
• Understanding Generative Recommendation with Semantic IDs from a Model-scaling View [57.471604518714535]
  Generative Recommendation (GR) tries to unify rich item semantics and collaborative filtering signals.<n>One popular modern approach is to use semantic IDs (SIDs) to represent items in an autoregressive user interaction sequence modeling setup.<n>We show that SID-based GR shows significant bottlenecks while scaling up the model.<n>We revisit another GR paradigm that directly uses large language models (LLMs) as recommenders.
  arXiv  Detail & Related papers  (2025-09-29T21:24:17Z)
• Reinforced Model Merging [53.84354455400038]
  We present an innovative framework termed Reinforced Model Merging (RMM), which encompasses an environment and agent tailored for merging tasks.<n>By utilizing data subsets during the evaluation process, we addressed the bottleneck in the reward feedback phase, thereby accelerating RMM by up to 100 times.
  arXiv  Detail & Related papers  (2025-03-27T08:52:41Z)
• MCTS-RAG: Enhancing Retrieval-Augmented Generation with Monte Carlo Tree Search [61.11836311160951]
  We introduce MCTS-RAG, a novel approach that enhances the reasoning capabilities of small language models on knowledge-intensive tasks.<n>Unlike standard RAG methods, which typically retrieve information independently from reasoning, MCTS-RAG combines structured reasoning with adaptive retrieval.<n>This integrated approach enhances decision-making, reduces hallucinations, and ensures improved factual accuracy and response consistency.
  arXiv  Detail & Related papers  (2025-03-26T17:46:08Z)
• Single Domain Generalization with Model-aware Parametric Batch-wise Mixup [22.709796153794507]
  Single Domain Generalization remains a formidable challenge in the field of machine learning.<n>We propose a novel data augmentation approach, named as Model-aware Parametric Batch-wise Mixup.<n>By exploiting inter-feature correlations, the parameterized mixup generator introduces additional versatility in combining features across a batch of instances.
  arXiv  Detail & Related papers  (2025-02-22T03:45:18Z)
• DPGIIL: Dirichlet Process-Deep Generative Model-Integrated Incremental Learning for Clustering in Transmissibility-based Online Structural Anomaly Detection [0.34376560669160394]
  This work proposes a novel clustering framework, referred to as Dirichlet process-deep generative model-integrated incremental learning (DPGIIL)<n>For online structural anomaly detection, DPGIIL can not only detect anomalies by dynamically assigning incoming data to new clusters but also indicate different structural states using distinct clusters.
  arXiv  Detail & Related papers  (2024-12-06T05:18:58Z)
• Pistis-RAG: Enhancing Retrieval-Augmented Generation with Human Feedback [41.88662700261036]
  RAG systems face limitations when semantic relevance alone does not guarantee improved generation quality. We propose Pistis-RAG, a new RAG framework designed with a content-centric approach to better align LLMs with human preferences.
  arXiv  Detail & Related papers  (2024-06-21T08:52:11Z)
• Exploring User Retrieval Integration towards Large Language Models for Cross-Domain Sequential Recommendation [66.72195610471624]
  Cross-Domain Sequential Recommendation aims to mine and transfer users' sequential preferences across different domains. We propose a novel framework named URLLM, which aims to improve the CDSR performance by exploring the User Retrieval approach.
  arXiv  Detail & Related papers  (2024-06-05T09:19:54Z)
• Task Groupings Regularization: Data-Free Meta-Learning with Heterogeneous Pre-trained Models [83.02797560769285]
  Data-Free Meta-Learning (DFML) aims to derive knowledge from a collection of pre-trained models without accessing their original data.<n>Current methods often overlook the heterogeneity among pre-trained models, which leads to performance degradation due to task conflicts.
  arXiv  Detail & Related papers  (2024-05-26T13:11:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.