Deep Domain Specialisation for single-model multi-domain learning to rank

• URL: http://arxiv.org/abs/2407.01069v1
• Date: Mon, 01 Jul 2024 08:19:19 GMT
• Title: Deep Domain Specialisation for single-model multi-domain learning to rank
• Authors: Paul Missault, Abdelmaseeh Felfel,
• Abstract summary: Training multiple models comes at a higher cost to train, maintain and update compared to having only a single model responsible for all domains. We propose a novel architecture of Deep Domain Specialisation (DDS) to consolidate multiple domains into a single model.
• Score: 1.534667887016089
• License:
• Abstract: Information Retrieval (IR) practitioners often train separate ranking models for different domains (geographic regions, languages, stores, websites,...) as it is believed that exclusively training on in-domain data yields the best performance when sufficient data is available. Despite their performance gains, training multiple models comes at a higher cost to train, maintain and update compared to having only a single model responsible for all domains. Our work explores consolidated ranking models that serve multiple domains. Specifically, we propose a novel architecture of Deep Domain Specialisation (DDS) to consolidate multiple domains into a single model. We compare our proposal against Deep Domain Adaptation (DDA) and a set of baseline for multi-domain models. In our experiments, DDS performed the best overall while requiring fewer parameters per domain as other baselines. We show the efficacy of our method both with offline experimentation and on a large-scale online experiment on Amazon customer traffic.

Related papers

• Virtual Classification: Modulating Domain-Specific Knowledge for Multidomain Crowd Counting [67.38137379297717]
  Multidomain crowd counting aims to learn a general model for multiple diverse datasets. Deep networks prefer modeling distributions of the dominant domains instead of all domains, which is known as domain bias. We propose a Modulating Domain-specific Knowledge Network (MDKNet) to handle the domain bias issue in multidomain crowd counting.
  arXiv  Detail & Related papers  (2024-02-06T06:49:04Z)
• Decoupled Training: Return of Frustratingly Easy Multi-Domain Learning [20.17925272562433]
  Multi-domain learning aims to train a model with minimal average risk across multiple overlapping but non-identical domains. We propose Decoupled Training (D-Train) as a frustratingly easy and hyper parameter-free multi-domain learning method. D-Train is a tri-phase general-to-specific training strategy that first pre-trains on all domains to warm up a root model, then post-trains on each domain by splitting into multi-heads, and finally fine-tunes the heads by fixing the backbone.
  arXiv  Detail & Related papers  (2023-09-19T04:06:41Z)
• Adapting Self-Supervised Representations to Multi-Domain Setups [47.03992469282679]
  Current state-of-the-art self-supervised approaches, are effective when trained on individual domains but show limited generalization on unseen domains. We propose a general-purpose, lightweight Domain Disentanglement Module that can be plugged into any self-supervised encoder.
  arXiv  Detail & Related papers  (2023-09-07T20:05:39Z)
• Exploring Distributional Shifts in Large Language Models for Code Analysis [36.73114441988879]
  We study how three large language models with code capabilities generalize to out-of-domain data. We consider two fundamental applications - code summarization, and code generation. We find that a model adapted to multiple domains simultaneously performs on par with those adapted to a single domain.
  arXiv  Detail & Related papers  (2023-03-16T07:45:46Z)
• Meta-DMoE: Adapting to Domain Shift by Meta-Distillation from Mixture-of-Experts [33.21435044949033]
  Most existing methods perform training on multiple source domains using a single model. We propose a novel framework for unsupervised test-time adaptation, which is formulated as a knowledge distillation process.
  arXiv  Detail & Related papers  (2022-10-08T02:28:10Z)
• Disentangled Modeling of Domain and Relevance for Adaptable Dense Retrieval [54.349418995689284]
  We propose a novel Dense Retrieval (DR) framework named Disentangled Dense Retrieval ( DDR) to support effective domain adaptation for DR models. By making the REM and DAMs disentangled, DDR enables a flexible training paradigm in which REM is trained with supervision once and DAMs are trained with unsupervised data. DDR significantly improves ranking performance compared to strong DR baselines and substantially outperforms traditional retrieval methods in most scenarios.
  arXiv  Detail & Related papers  (2022-08-11T11:18:50Z)
• Variational Attention: Propagating Domain-Specific Knowledge for Multi-Domain Learning in Crowd Counting [75.80116276369694]
  In crowd counting, due to the problem of laborious labelling, it is perceived intractability of collecting a new large-scale dataset. We resort to the multi-domain joint learning and propose a simple but effective Domain-specific Knowledge Propagating Network (DKPNet) It is mainly achieved by proposing the novel Variational Attention(VA) technique for explicitly modeling the attention distributions for different domains.
  arXiv  Detail & Related papers  (2021-08-18T08:06:37Z)
• Batch Normalization Embeddings for Deep Domain Generalization [50.51405390150066]
  Domain generalization aims at training machine learning models to perform robustly across different and unseen domains. We show a significant increase in classification accuracy over current state-of-the-art techniques on popular domain generalization benchmarks.
  arXiv  Detail & Related papers  (2020-11-25T12:02:57Z)
• Multi-Domain Spoken Language Understanding Using Domain- and Task-Aware Parameterization [78.93669377251396]
  Spoken language understanding has been addressed as a supervised learning problem, where a set of training data is available for each domain. One existing approach solves the problem by conducting multi-domain learning, using shared parameters for joint training across domains. We propose to improve the parameterization of this method by using domain-specific and task-specific model parameters.
  arXiv  Detail & Related papers  (2020-04-30T15:15:40Z)
• Dynamic Fusion Network for Multi-Domain End-to-end Task-Oriented Dialog [70.79442700890843]
  We propose a novel Dynamic Fusion Network (DF-Net) which automatically exploit the relevance between the target domain and each domain. With little training data, we show its transferability by outperforming prior best model by 13.9% on average.
  arXiv  Detail & Related papers  (2020-04-23T08:17:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.