An Unified Search and Recommendation Foundation Model for Cold-Start
Scenario
- URL: http://arxiv.org/abs/2309.08939v1
- Date: Sat, 16 Sep 2023 10:00:02 GMT
- Title: An Unified Search and Recommendation Foundation Model for Cold-Start
Scenario
- Authors: Yuqi Gong, Xichen Ding, Yehui Su, Kaiming Shen, Zhongyi Liu, Guannan
Zhang
- Abstract summary: In commercial search engines and recommendation systems, data from multiple domains is available to jointly train the multi-domain model.
We propose a novel framework called S&R Multi-Domain Foundation, which uses LLM to extract domain invariant features.
We apply the S&R Multi-Domain foundation model to cold start scenarios in the pretrain-finetune manner, which achieves better performance than other SOTA transfer learning methods.
- Score: 15.192845741415738
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: In modern commercial search engines and recommendation systems, data from
multiple domains is available to jointly train the multi-domain model.
Traditional methods train multi-domain models in the multi-task setting, with
shared parameters to learn the similarity of multiple tasks, and task-specific
parameters to learn the divergence of features, labels, and sample
distributions of individual tasks. With the development of large language
models, LLM can extract global domain-invariant text features that serve both
search and recommendation tasks. We propose a novel framework called S\&R
Multi-Domain Foundation, which uses LLM to extract domain invariant features,
and Aspect Gating Fusion to merge the ID feature, domain invariant text
features and task-specific heterogeneous sparse features to obtain the
representations of query and item. Additionally, samples from multiple search
and recommendation scenarios are trained jointly with Domain Adaptive
Multi-Task module to obtain the multi-domain foundation model. We apply the
S\&R Multi-Domain foundation model to cold start scenarios in the
pretrain-finetune manner, which achieves better performance than other SOTA
transfer learning methods. The S\&R Multi-Domain Foundation model has been
successfully deployed in Alipay Mobile Application's online services, such as
content query recommendation and service card recommendation, etc.
Related papers
- Investigating the potential of Sparse Mixtures-of-Experts for multi-domain neural machine translation [59.41178047749177]
We focus on multi-domain Neural Machine Translation, with the goal of developing efficient models which can handle data from various domains seen during training and are robust to domains unseen during training.
We hypothesize that Sparse Mixture-of-Experts (SMoE) models are a good fit for this task, as they enable efficient model scaling.
We conduct a series of experiments aimed at validating the utility of SMoE for the multi-domain scenario, and find that a straightforward width scaling of Transformer is a simpler and surprisingly more efficient approach in practice, and reaches the same performance level as SMoE.
arXiv Detail & Related papers (2024-07-01T09:45:22Z) - Exploring User Retrieval Integration towards Large Language Models for Cross-Domain Sequential Recommendation [66.72195610471624]
Cross-Domain Sequential Recommendation aims to mine and transfer users' sequential preferences across different domains.
We propose a novel framework named URLLM, which aims to improve the CDSR performance by exploring the User Retrieval approach.
arXiv Detail & Related papers (2024-06-05T09:19:54Z) - M3oE: Multi-Domain Multi-Task Mixture-of Experts Recommendation Framework [32.68911775382326]
M3oE is an adaptive Multi-domain Multi-task Mixture-of-Experts recommendation framework.
We leverage three mixture-of-experts modules to learn common, domain-aspect, and task-aspect user preferences.
We design a two-level fusion mechanism for precise control over feature extraction and fusion across diverse domains and tasks.
arXiv Detail & Related papers (2024-04-29T06:59:30Z) - Multi-BERT: Leveraging Adapters and Prompt Tuning for Low-Resource Multi-Domain Adaptation [14.211024633768986]
The rapid expansion of texts' volume and diversity presents formidable challenges in multi-domain settings.
Traditional approaches, either employing a unified model for multiple domains or individual models for each domain, frequently pose significant limitations.
This paper introduces a novel approach composed of one core model with multiple sets of domain-specific parameters.
arXiv Detail & Related papers (2024-04-02T22:15:48Z) - Exploring Distributional Shifts in Large Language Models for Code
Analysis [36.73114441988879]
We study how three large language models with code capabilities generalize to out-of-domain data.
We consider two fundamental applications - code summarization, and code generation.
We find that a model adapted to multiple domains simultaneously performs on par with those adapted to a single domain.
arXiv Detail & Related papers (2023-03-16T07:45:46Z) - Multi-Modal Cross-Domain Alignment Network for Video Moment Retrieval [55.122020263319634]
Video moment retrieval (VMR) aims to localize the target moment from an untrimmed video according to a given language query.
In this paper, we focus on a novel task: cross-domain VMR, where fully-annotated datasets are available in one domain but the domain of interest only contains unannotated datasets.
We propose a novel Multi-Modal Cross-Domain Alignment network to transfer the annotation knowledge from the source domain to the target domain.
arXiv Detail & Related papers (2022-09-23T12:58:20Z) - Multi-path Neural Networks for On-device Multi-domain Visual
Classification [55.281139434736254]
This paper proposes a novel approach to automatically learn a multi-path network for multi-domain visual classification on mobile devices.
The proposed multi-path network is learned from neural architecture search by applying one reinforcement learning controller for each domain to select the best path in the super-network created from a MobileNetV3-like search space.
The determined multi-path model selectively shares parameters across domains in shared nodes while keeping domain-specific parameters within non-shared nodes in individual domain paths.
arXiv Detail & Related papers (2020-10-10T05:13:49Z) - Learning to Combine: Knowledge Aggregation for Multi-Source Domain
Adaptation [56.694330303488435]
We propose a Learning to Combine for Multi-Source Domain Adaptation (LtC-MSDA) framework.
In the nutshell, a knowledge graph is constructed on the prototypes of various domains to realize the information propagation among semantically adjacent representations.
Our approach outperforms existing methods with a remarkable margin.
arXiv Detail & Related papers (2020-07-17T07:52:44Z) - Mutual Learning Network for Multi-Source Domain Adaptation [73.25974539191553]
We propose a novel multi-source domain adaptation method, Mutual Learning Network for Multiple Source Domain Adaptation (ML-MSDA)
Under the framework of mutual learning, the proposed method pairs the target domain with each single source domain to train a conditional adversarial domain adaptation network as a branch network.
The proposed method outperforms the comparison methods and achieves the state-of-the-art performance.
arXiv Detail & Related papers (2020-03-29T04:31:43Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.