MLoRA: Multi-Domain Low-Rank Adaptive Network for CTR Prediction

• URL: http://arxiv.org/abs/2408.08913v1
• Date: Wed, 14 Aug 2024 05:53:02 GMT
• Title: MLoRA: Multi-Domain Low-Rank Adaptive Network for CTR Prediction
• Authors: Zhiming Yang, Haining Gao, Dehong Gao, Luwei Yang, Libin Yang, Xiaoyan Cai, Wei Ning, Guannan Zhang,
• Abstract summary: We propose a Multi-domain Low-Rank Adaptive network (MLoRA) for CTR prediction, where we introduce a specialized LoRA module for each domain. Experimental results demonstrate our MLoRA approach achieves a significant improvement compared with state-of-the-art baselines. The code of our MLoRA is publicly available.
• Score: 18.524017579108044
• License:
• Abstract: Click-through rate (CTR) prediction is one of the fundamental tasks in the industry, especially in e-commerce, social media, and streaming media. It directly impacts website revenues, user satisfaction, and user retention. However, real-world production platforms often encompass various domains to cater for diverse customer needs. Traditional CTR prediction models struggle in multi-domain recommendation scenarios, facing challenges of data sparsity and disparate data distributions across domains. Existing multi-domain recommendation approaches introduce specific-domain modules for each domain, which partially address these issues but often significantly increase model parameters and lead to insufficient training. In this paper, we propose a Multi-domain Low-Rank Adaptive network (MLoRA) for CTR prediction, where we introduce a specialized LoRA module for each domain. This approach enhances the model's performance in multi-domain CTR prediction tasks and is able to be applied to various deep-learning models. We evaluate the proposed method on several multi-domain datasets. Experimental results demonstrate our MLoRA approach achieves a significant improvement compared with state-of-the-art baselines. Furthermore, we deploy it in the production environment of the Alibaba.COM. The online A/B testing results indicate the superiority and flexibility in real-world production environments. The code of our MLoRA is publicly available.

Related papers

• Deep Domain Specialisation for single-model multi-domain learning to rank [1.534667887016089]
  Training multiple models comes at a higher cost to train, maintain and update compared to having only a single model responsible for all domains. We propose a novel architecture of Deep Domain Specialisation (DDS) to consolidate multiple domains into a single model.
  arXiv  Detail & Related papers  (2024-07-01T08:19:19Z)
• Star+: A New Multi-Domain Model for CTR Prediction [0.0]
  We introduce Star+, a novel multi-domain model for click-through rate (CTR) prediction inspired by the Star model. Our experiments on both industrial and public datasets demonstrate that Star+ significantly improves prediction accuracy and efficiency.
  arXiv  Detail & Related papers  (2024-06-24T12:03:35Z)
• Crocodile: Cross Experts Covariance for Disentangled Learning in Multi-Domain Recommendation [23.010588147317623]
  We propose a novel Cross-experts Covariance Loss for Disentangled Learning model (Crocodile) It employs multiple embedding tables to make the model domain-aware at the embeddings which consist most parameters in the model. Crocodile achieves 0.72% CTR lift and 0.73% GMV lift on a primary advertising scenario.
  arXiv  Detail & Related papers  (2024-05-21T11:54:16Z)
• Role Prompting Guided Domain Adaptation with General Capability Preserve for Large Language Models [55.51408151807268]
  When tailored to specific domains, Large Language Models (LLMs) tend to experience catastrophic forgetting. crafting a versatile model for multiple domains simultaneously often results in a decline in overall performance. We present the RolE Prompting Guided Multi-Domain Adaptation (REGA) strategy.
  arXiv  Detail & Related papers  (2024-03-05T08:22:41Z)
• Virtual Classification: Modulating Domain-Specific Knowledge for Multidomain Crowd Counting [67.38137379297717]
  Multidomain crowd counting aims to learn a general model for multiple diverse datasets. Deep networks prefer modeling distributions of the dominant domains instead of all domains, which is known as domain bias. We propose a Modulating Domain-specific Knowledge Network (MDKNet) to handle the domain bias issue in multidomain crowd counting.
  arXiv  Detail & Related papers  (2024-02-06T06:49:04Z)
• Improving Domain Generalization with Domain Relations [77.63345406973097]
  This paper focuses on domain shifts, which occur when the model is applied to new domains that are different from the ones it was trained on. We propose a new approach called D$3$G to learn domain-specific models. Our results show that D$3$G consistently outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-02-06T08:11:16Z)
• AdaptDHM: Adaptive Distribution Hierarchical Model for Multi-Domain CTR Prediction [4.299153274884263]
  We propose an elegant and flexible multi-distribution modeling paradigm, named Adaptive Distribution Hierarchical Model (AdaptDHM) Our model achieves impressive prediction accuracy and its time cost during the training stage is more than 50% less than that of other models.
  arXiv  Detail & Related papers  (2022-11-22T09:10:37Z)
• META: Mimicking Embedding via oThers' Aggregation for Generalizable Person Re-identification [68.39849081353704]
  Domain generalizable (DG) person re-identification (ReID) aims to test across unseen domains without access to the target domain data at training time. This paper presents a new approach called Mimicking Embedding via oThers' Aggregation (META) for DG ReID.
  arXiv  Detail & Related papers  (2021-12-16T08:06:50Z)
• Variational Attention: Propagating Domain-Specific Knowledge for Multi-Domain Learning in Crowd Counting [75.80116276369694]
  In crowd counting, due to the problem of laborious labelling, it is perceived intractability of collecting a new large-scale dataset. We resort to the multi-domain joint learning and propose a simple but effective Domain-specific Knowledge Propagating Network (DKPNet) It is mainly achieved by proposing the novel Variational Attention(VA) technique for explicitly modeling the attention distributions for different domains.
  arXiv  Detail & Related papers  (2021-08-18T08:06:37Z)
• Dual Attentive Sequential Learning for Cross-Domain Click-Through Rate Prediction [76.98616102965023]
  Cross domain recommender system constitutes a powerful method to tackle the cold-start and sparsity problem. We propose a novel approach to cross-domain sequential recommendations based on the dual learning mechanism.
  arXiv  Detail & Related papers  (2021-06-05T01:21:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.