Adaptive Conditional Expert Selection Network for Multi-domain Recommendation

• URL: http://arxiv.org/abs/2411.06826v1
• Date: Mon, 11 Nov 2024 09:39:31 GMT
• Title: Adaptive Conditional Expert Selection Network for Multi-domain Recommendation
• Authors: Kuiyao Dong, Xingyu Lou, Feng Liu, Ruian Wang, Wenyi Yu, Ping Wang, Jun Wang,
• Abstract summary: Mixture-of-Experts (MOE) has recently become the de facto standard in Multi-domain recommendation (MDR) CESAA consists of Conditional Expert Selection (CES) Module and Adaptive Expert Aggregation (AEA) Module. AEA utilizes mutual information loss to strengthen the correlations between experts and specific domains, and significantly improve the distinction between experts.
• Score: 10.418133538132635
• License:
• Abstract: Mixture-of-Experts (MOE) has recently become the de facto standard in Multi-domain recommendation (MDR) due to its powerful expressive ability. However, such MOE-based method typically employs all experts for each instance, leading to scalability issue and low-discriminability between domains and experts. Furthermore, the design of commonly used domain-specific networks exacerbates the scalability issues. To tackle the problems, We propose a novel method named CESAA consists of Conditional Expert Selection (CES) Module and Adaptive Expert Aggregation (AEA) Module to tackle these challenges. Specifically, CES first combines a sparse gating strategy with domain-shared experts. Then AEA utilizes mutual information loss to strengthen the correlations between experts and specific domains, and significantly improve the distinction between experts. As a result, only domain-shared experts and selected domain-specific experts are activated for each instance, striking a balance between computational efficiency and model performance. Experimental results on both public ranking and industrial retrieval datasets verify the effectiveness of our method in MDR tasks.

Related papers

• LFME: A Simple Framework for Learning from Multiple Experts in Domain Generalization [61.16890890570814]
  Domain generalization (DG) methods aim to maintain good performance in an unseen target domain by using training data from multiple source domains. This work introduces a simple yet effective framework, dubbed learning from multiple experts (LFME) that aims to make the target model an expert in all source domains to improve DG.
  arXiv  Detail & Related papers  (2024-10-22T13:44:10Z)
• Flexible and Adaptable Summarization via Expertise Separation [59.26639426529827]
  A proficient summarization model should exhibit both flexibility and adaptability. We propose MoeSumm, a Mixture-of-Expert Summarization architecture. Our model's distinct separation of general and domain-specific summarization abilities grants it with notable flexibility and adaptability.
  arXiv  Detail & Related papers  (2024-06-08T05:31:19Z)
• Generalization Error Analysis for Sparse Mixture-of-Experts: A Preliminary Study [65.11303133775857]
  Mixture-of-Experts (MoE) computation amalgamates predictions from several specialized sub-models (referred to as experts) Sparse MoE selectively engages only a limited number, or even just one expert, significantly reducing overhead while empirically preserving, and sometimes even enhancing, performance.
  arXiv  Detail & Related papers  (2024-03-26T05:48:02Z)
• BECoTTA: Input-dependent Online Blending of Experts for Continual Test-time Adaptation [59.1863462632777]
  Continual Test Time Adaptation (CTTA) is required to adapt efficiently to continuous unseen domains while retaining previously learned knowledge. This paper proposes BECoTTA, an input-dependent and efficient modular framework for CTTA. We validate that our method outperforms multiple CTTA scenarios, including disjoint and gradual domain shits, while only requiring 98% fewer trainable parameters.
  arXiv  Detail & Related papers  (2024-02-13T18:37:53Z)
• Robust Representation Learning for Unified Online Top-K Recommendation [39.12191494863331]
  We propose a robust representation learning for the unified online top-k recommendation. Our approach constructs unified modeling in entity space to ensure data fairness. The proposed method has been successfully deployed online to serve real business scenarios.
  arXiv  Detail & Related papers  (2023-10-24T03:42:20Z)
• Adaptive Mixture of Experts Learning for Generalizable Face Anti-Spoofing [37.75738807247752]
  Face anti-spoofing approaches based on domain generalization (DG) have drawn growing attention. Existing DG-based FAS approaches always capture the domain-invariant features for generalizing on the various unseen domains. We propose an Adaptive Mixture of Experts Learning framework, which exploits the domain-specific information to adaptively establish the link among the seen source domains and unseen target domains.
  arXiv  Detail & Related papers  (2022-07-20T13:02:51Z)
• META: Mimicking Embedding via oThers' Aggregation for Generalizable Person Re-identification [68.39849081353704]
  Domain generalizable (DG) person re-identification (ReID) aims to test across unseen domains without access to the target domain data at training time. This paper presents a new approach called Mimicking Embedding via oThers' Aggregation (META) for DG ReID.
  arXiv  Detail & Related papers  (2021-12-16T08:06:50Z)
• Multiple Domain Experts Collaborative Learning: Multi-Source Domain Generalization For Person Re-Identification [41.923753462539736]
  We propose a novel training framework, named Multiple Domain Experts Collaborative Learning (MD-ExCo) The MD-ExCo consists of a universal expert and several domain experts. Experiments on DG-ReID benchmarks show that our MD-ExCo outperforms the state-of-the-art methods by a large margin.
  arXiv  Detail & Related papers  (2021-05-26T06:38:23Z)
• Multiple Expert Brainstorming for Domain Adaptive Person Re-identification [140.3998019639158]
  We propose a multiple expert brainstorming network (MEB-Net) for domain adaptive person re-ID. MEB-Net adopts a mutual learning strategy, where multiple networks with different architectures are pre-trained within a source domain. Experiments on large-scale datasets demonstrate the superior performance of MEB-Net over the state-of-the-arts.
  arXiv  Detail & Related papers  (2020-07-03T08:16:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.