M3oE: Multi-Domain Multi-Task Mixture-of Experts Recommendation Framework

• URL: http://arxiv.org/abs/2404.18465v3
• Date: Sun, 12 May 2024 13:11:29 GMT
• Title: M3oE: Multi-Domain Multi-Task Mixture-of Experts Recommendation Framework
• Authors: Zijian Zhang, Shuchang Liu, Jiaao Yu, Qingpeng Cai, Xiangyu Zhao, Chunxu Zhang, Ziru Liu, Qidong Liu, Hongwei Zhao, Lantao Hu, Peng Jiang, Kun Gai,
• Abstract summary: M3oE is an adaptive Multi-domain Multi-task Mixture-of-Experts recommendation framework. We leverage three mixture-of-experts modules to learn common, domain-aspect, and task-aspect user preferences. We design a two-level fusion mechanism for precise control over feature extraction and fusion across diverse domains and tasks.
• Score: 32.68911775382326
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-domain recommendation and multi-task recommendation have demonstrated their effectiveness in leveraging common information from different domains and objectives for comprehensive user modeling. Nonetheless, the practical recommendation usually faces multiple domains and tasks simultaneously, which cannot be well-addressed by current methods. To this end, we introduce M3oE, an adaptive Multi-domain Multi-task Mixture-of-Experts recommendation framework. M3oE integrates multi-domain information, maps knowledge across domains and tasks, and optimizes multiple objectives. We leverage three mixture-of-experts modules to learn common, domain-aspect, and task-aspect user preferences respectively to address the complex dependencies among multiple domains and tasks in a disentangled manner. Additionally, we design a two-level fusion mechanism for precise control over feature extraction and fusion across diverse domains and tasks. The framework's adaptability is further enhanced by applying AutoML technique, which allows dynamic structure optimization. To the best of the authors' knowledge, our M3oE is the first effort to solve multi-domain multi-task recommendation self-adaptively. Extensive experiments on two benchmark datasets against diverse baselines demonstrate M3oE's superior performance. The implementation code is available to ensure reproducibility.

Related papers

• Generative Multi-Target Cross-Domain Recommendation [48.54929268144516]
  This paper introduces GMC, a generative paradigm-based approach for multi-target cross-domain recommendation.<n>The core idea of GMC is to leverage semantically quantized discrete item identifiers as a medium for integrating multi-domain knowledge.<n>Extensive experiments on five public datasets demonstrate the effectiveness of GMC.
  arXiv  Detail & Related papers  (2025-07-17T07:44:05Z)
• LLM-Enhanced Multimodal Fusion for Cross-Domain Sequential Recommendation [19.654959889052638]
  Cross-Domain Sequential Recommendation (CDSR) predicts user behavior by leveraging historical interactions across multiple domains.<n>We propose LLM-Enhanced Multimodal Fusion for Cross-Domain Sequential Recommendation (LLM-EMF)<n>LLM-EMF is a novel and advanced approach that enhances textual information with Large Language Models (LLM) knowledge.
  arXiv  Detail & Related papers  (2025-06-22T09:53:21Z)
• MoE-MLoRA for Multi-Domain CTR Prediction: Efficient Adaptation with Expert Specialization [0.0]
  MoE-MLoRA is a mixture-of-experts framework where each expert is first trained independently to specialize in its domain.<n>We evaluate MoE-MLoRA across eight CTR models on Movielens and Taobao.
  arXiv  Detail & Related papers  (2025-06-09T09:03:05Z)
• IDEA: Inverted Text with Cooperative Deformable Aggregation for Multi-modal Object Re-Identification [60.38841251693781]
  We propose a novel framework to generate robust multi-modal object ReIDs. Our framework uses Modal Prefixes and InverseNet to integrate multi-modal information with semantic guidance from inverted text. Experiments on three multi-modal object ReID benchmarks demonstrate the effectiveness of our proposed method.
  arXiv  Detail & Related papers  (2025-03-13T13:00:31Z)
• Benchmarking Retrieval-Augmented Generation in Multi-Modal Contexts [56.30364248231053]
  This paper introduces Multi-Modal Retrieval-Augmented Generation (M2RAG) M2RAG is a benchmark designed to evaluate the effectiveness of Multi-modal Large Language Models (MLLMs) To enhance the context utilization capabilities of MLLMs, we also introduce Multi-Modal Retrieval-Augmented Instruction Tuning (MM-RAIT)
  arXiv  Detail & Related papers  (2025-02-24T16:25:25Z)
• Joint Similarity Item Exploration and Overlapped User Guidance for Multi-Modal Cross-Domain Recommendation [27.00142195880019]
  We propose Joint Similarity Item Exploration and Overlapped User Guidance (SIEOUG) for solving the Multi-Modal Cross-Domain Recommendation problem. Our empirical study on Amazon dataset with several different tasks demonstrates that SIEOUG significantly outperforms the state-of-the-art models under the MMCDR setting.
  arXiv  Detail & Related papers  (2025-02-22T03:57:43Z)
• MDAP: A Multi-view Disentangled and Adaptive Preference Learning Framework for Cross-Domain Recommendation [63.27390451208503]
  Cross-domain Recommendation systems leverage multi-domain user interactions to improve performance. We propose the Multi-view Disentangled and Adaptive Preference Learning framework. Our framework uses a multiview encoder to capture diverse user preferences.
  arXiv  Detail & Related papers  (2024-10-08T10:06:45Z)
• Ducho meets Elliot: Large-scale Benchmarks for Multimodal Recommendation [9.506245109666907]
  Multi-faceted features characterizing products and services may influence each customer on online selling platforms differently. The common multimodal recommendation pipeline involves (i) extracting multimodal features, (ii) refining their high-level representations to suit the recommendation task, and (iv) predicting the user-item score. This paper settles as the first attempt to offer a large-scale benchmarking for multimodal recommender systems, with a specific focus on multimodal extractors.
  arXiv  Detail & Related papers  (2024-09-24T08:29:10Z)
• Towards Personalized Federated Multi-Scenario Multi-Task Recommendation [22.095138650857436]
  PF-MSMTrec is a novel framework for personalized federated multi-scenario multi-task recommendation. We introduce a bottom-up joint learning mechanism to address the unique challenges of multiple optimization conflicts. Our proposed method outperforms state-of-the-art approaches.
  arXiv  Detail & Related papers  (2024-06-27T07:10:37Z)
• Uni-MoE: Scaling Unified Multimodal LLMs with Mixture of Experts [54.529880848937104]
  We develop a unified MLLM with the MoE architecture, named Uni-MoE, that can handle a wide array of modalities. Specifically, it features modality-specific encoders with connectors for a unified multimodal representation. We evaluate the instruction-tuned Uni-MoE on a comprehensive set of multimodal datasets.
  arXiv  Detail & Related papers  (2024-05-18T12:16:01Z)
• Intuition-aware Mixture-of-Rank-1-Experts for Parameter Efficient Finetuning [50.73666458313015]
  Large Language Models (LLMs) have demonstrated significant potential in performing multiple tasks in multimedia applications. MoE has been emerged as a promising solution with its sparse architecture for effective task decoupling. Intuition-MoR1E achieves superior efficiency and 2.15% overall accuracy improvement across 14 public datasets.
  arXiv  Detail & Related papers  (2024-04-13T12:14:58Z)
• AlignRec: Aligning and Training in Multimodal Recommendations [29.995007279325947]
  multimodal recommendations can leverage rich contexts beyond interactions. Existing methods mainly regard multimodal information as an auxiliary, using them to help learn ID features. There exist semantic gaps among multimodal content features and ID-based features, for which directly using multimodal information as an auxiliary would lead to misalignment in representations of users and items.
  arXiv  Detail & Related papers  (2024-03-19T02:49:32Z)
• Mixture-of-LoRAs: An Efficient Multitask Tuning for Large Language Models [7.966452497550907]
  We propose the Mixture-of-LoRAs (MoA) architecture for multi-task learning with large language models (LLMs) Multiple domain-specific LoRA modules can be aligned with the expert design principles observed in Mixture-of-Experts (MoE) Each LoRA model can be iteratively adapted to a new domain, allowing for quick domain-specific adaptation.
  arXiv  Detail & Related papers  (2024-03-06T03:33:48Z)
• Multimodal Instruction Tuning with Conditional Mixture of LoRA [54.65520214291653]
  This paper introduces a novel approach that integrates multimodal instruction tuning with Low-Rank Adaption (LoRA) It innovates upon LoRA by dynamically constructing low-rank adaptation matrices tailored to the unique demands of each input instance. Experimental results on various multimodal evaluation datasets indicate that MixLoRA not only outperforms the conventional LoRA with the same or even higher ranks.
  arXiv  Detail & Related papers  (2024-02-24T20:15:31Z)
• An Unified Search and Recommendation Foundation Model for Cold-Start Scenario [15.192845741415738]
  In commercial search engines and recommendation systems, data from multiple domains is available to jointly train the multi-domain model. We propose a novel framework called S&R Multi-Domain Foundation, which uses LLM to extract domain invariant features. We apply the S&R Multi-Domain foundation model to cold start scenarios in the pretrain-finetune manner, which achieves better performance than other SOTA transfer learning methods.
  arXiv  Detail & Related papers  (2023-09-16T10:00:02Z)
• Dual Metric Learning for Effective and Efficient Cross-Domain Recommendations [85.6250759280292]
  Cross domain recommender systems have been increasingly valuable for helping consumers identify useful items in different applications. Existing cross-domain models typically require large number of overlap users, which can be difficult to obtain in some applications. We propose a novel cross-domain recommendation model based on dual learning that transfers information between two related domains in an iterative manner.
  arXiv  Detail & Related papers  (2021-04-17T09:18:59Z)
• Multi-path Neural Networks for On-device Multi-domain Visual Classification [55.281139434736254]
  This paper proposes a novel approach to automatically learn a multi-path network for multi-domain visual classification on mobile devices. The proposed multi-path network is learned from neural architecture search by applying one reinforcement learning controller for each domain to select the best path in the super-network created from a MobileNetV3-like search space. The determined multi-path model selectively shares parameters across domains in shared nodes while keeping domain-specific parameters within non-shared nodes in individual domain paths.
  arXiv  Detail & Related papers  (2020-10-10T05:13:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.