Distillation-based Scenario-Adaptive Mixture-of-Experts for the Matching Stage of Multi-scenario Recommendation

• URL: http://arxiv.org/abs/2601.02368v1
• Date: Fri, 28 Nov 2025 12:04:29 GMT
• Title: Distillation-based Scenario-Adaptive Mixture-of-Experts for the Matching Stage of Multi-scenario Recommendation
• Authors: Ruibing Wang, Shuhan Guo, Haotong Du, Quanming Yao,
• Abstract summary: We propose Distillation-based Scenario-Adaptive Mixture-of-Experts (DSMOE)<n> Specially, we devise a Scenario-Adaptive Projection (SAP) module to generate lightweight, context-specific parameters.<n>We introduce a cross-architecture knowledge distillation framework, where an interaction-aware teacher guides the two-tower student to capture complex matching patterns.
• Score: 29.435021078824064
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-scenario recommendation is pivotal for optimizing user experience across diverse contexts. While Multi-gate Mixture-of-Experts (MMOE) thrives in ranking, its transfer to the matching stage is hindered by the blind optimization inherent to independent two-tower architectures and the parameter dominance of head scenarios. To address these structural and distributional bottlenecks, we propose Distillation-based Scenario-Adaptive Mixture-of-Experts (DSMOE). Specially, we devise a Scenario-Adaptive Projection (SAP) module to generate lightweight, context-specific parameters, effectively preventing expert collapse in long-tail scenarios. Concurrently, we introduce a cross-architecture knowledge distillation framework, where an interaction-aware teacher guides the two-tower student to capture complex matching patterns. Extensive experiments on real-world datasets demonstrate DSMOE's superiority, particularly in significantly improving retrieval quality for under-represented, data-sparse scenarios.

Related papers

• Mixture of Predefined Experts: Maximizing Data Usage on Vertical Federated Learning [4.1346825639910785]
  Split-MoPE is a novel framework that integrates Split Learning with a specialized Mixture of Predefined Experts architecture.<n>MoPE uses predefined experts to process specific data alignments, effectively maximizing data usage during both training and inference.<n>By leveraging pretrained encoders for target data domains, Split-MoPE achieves state-of-the-art performance in a single communication round.
  arXiv  Detail & Related papers  (2026-02-13T08:21:20Z)
• Multifaceted Scenario-Aware Hypergraph Learning for Next POI Recommendation [6.180520055741916]
  Next Point-of-Interest (POI) recommendation plays a crucial role in inferring user preferences from historical check-in trajectories.<n>Existing sequential and graph-based methods frequently neglect significant mobility variations across distinct contextual scenarios.<n>We propose the Multifaceted Scenario-Aware Hypergraph Learning method (MSAHG), a framework that adopts a scenario-splitting paradigm for next POI recommendation.
  arXiv  Detail & Related papers  (2026-01-09T06:29:55Z)
• Multi-Level Aware Preference Learning: Enhancing RLHF for Complex Multi-Instruction Tasks [81.44256822500257]
  RLHF has emerged as a predominant approach for aligning artificial intelligence systems with human preferences.<n> RLHF exhibits insufficient compliance capabilities when confronted with complex multi-instruction tasks.<n>We propose a novel Multi-level Aware Preference Learning (MAPL) framework, capable of enhancing multi-instruction capabilities.
  arXiv  Detail & Related papers  (2025-05-19T08:33:11Z)
• Towards Generalizable Trajectory Prediction Using Dual-Level Representation Learning And Adaptive Prompting [107.4034346788744]
  Existing vehicle trajectory prediction models struggle with generalizability, prediction uncertainties, and handling complex interactions.<n>We propose Perceiver with Register queries (PerReg+), a novel trajectory prediction framework that introduces: (1) Dual-Level Representation Learning via Self-Distillation (SD) and Masked Reconstruction (MR), capturing global context and fine-grained details; (2) Enhanced Multimodality using register-based queries and pretraining, eliminating the need for clustering and suppression; and (3) Adaptive Prompt Tuning during fine-tuning, freezing the main architecture and optimizing a small number of prompts for efficient adaptation.
  arXiv  Detail & Related papers  (2025-01-08T20:11:09Z)
• Scenario-Wise Rec: A Multi-Scenario Recommendation Benchmark [65.13288661320364]
  We introduce our benchmark, textbfScenario-Wise Rec, which comprises 6 public datasets and 12 benchmark models, along with a training and evaluation pipeline.<n>We aim for this benchmark to offer researchers valuable insights from prior work, enabling the development of novel models.
  arXiv  Detail & Related papers  (2024-12-23T08:15:34Z)
• A Systematic Examination of Preference Learning through the Lens of Instruction-Following [83.71180850955679]
  We use a novel synthetic data generation pipeline to generate 48,000 instruction unique-following prompts.<n>With our synthetic prompts, we use two preference dataset curation methods - rejection sampling (RS) and Monte Carlo Tree Search (MCTS)<n>Experiments reveal that shared prefixes in preference pairs, as generated by MCTS, provide marginal but consistent improvements.<n>High-contrast preference pairs generally outperform low-contrast pairs; however, combining both often yields the best performance.
  arXiv  Detail & Related papers  (2024-12-18T15:38:39Z)
• MITA: Bridging the Gap between Model and Data for Test-time Adaptation [68.62509948690698]
  Test-Time Adaptation (TTA) has emerged as a promising paradigm for enhancing the generalizability of models. We propose Meet-In-The-Middle based MITA, which introduces energy-based optimization to encourage mutual adaptation of the model and data from opposing directions.
  arXiv  Detail & Related papers  (2024-10-12T07:02:33Z)
• Adaptive Utilization of Cross-scenario Information for Multi-scenario Recommendation [11.489766641148151]
  Multi-scenario Recommendation (MSR) is an important topic that improves ranking performance by leveraging information from different scenarios. Recent methods for MSR mostly construct scenario shared or specific modules to model commonalities and differences among scenarios. We propose a unified model named Cross-Scenario Information Interaction (CSII) to serve all scenarios by a mixture of scenario-dominated experts.
  arXiv  Detail & Related papers  (2024-07-29T06:17:33Z)
• LLM4MSR: An LLM-Enhanced Paradigm for Multi-Scenario Recommendation [52.55639178180821]
  The study on multi-scenario recommendation (MSR) has attracted much attention, which uses the data from all scenarios to simultaneously improve their recommendation performance.<n>Existing methods tend to integrate insufficient scenario knowledge and neglect learning personalized cross-scenario preferences, thus leading to sub-optimal performance.<n>We propose a large language model (LLM)-enhanced paradigm LLM4MSR to fill these gaps.
  arXiv  Detail & Related papers  (2024-06-18T11:59:36Z)
• Pareto Driven Surrogate (ParDen-Sur) Assisted Optimisation of Multi-period Portfolio Backtest Simulations [0.0]
  This study presents the glsParDen-Sur modelling framework to efficiently perform the required hyper- parameter search. glsParDen-Sur extends previous surrogate frameworks by including a reservoir sampling-based look-ahead mechanism for offspring generation in glsplEA alongside the traditional acceptance sampling scheme.
  arXiv  Detail & Related papers  (2022-09-13T07:29:20Z)
• Scenario-Adaptive and Self-Supervised Model for Multi-Scenario Personalized Recommendation [35.4495536683099]
  We propose a scenario-Adaptive and Self-Supervised (SASS) model to solve the three challenges mentioned above. The model is created symmetrically both in user side and item side, so that we can get distinguishing representations of items in different scenarios. This model also achieves more than 8.0% improvement on Average Watching Time Per User in online A/B tests.
  arXiv  Detail & Related papers  (2022-08-24T11:44:00Z)
• Scenario-aware and Mutual-based approach for Multi-scenario Recommendation in E-Commerce [12.794276204716642]
  How to make accurate recommendations for users in heterogeneous e-commerce scenarios is still a continuous research topic. We propose a novel recommendation model named Scenario-aware Mutual Learning (SAML) that leverages the differences and similarities between multiple scenarios.
  arXiv  Detail & Related papers  (2020-12-16T13:52:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.