Efficient Large-Scale Cross-Domain Sequential Recommendation with Dynamic State Representations

• URL: http://arxiv.org/abs/2508.20945v1
• Date: Thu, 28 Aug 2025 16:05:42 GMT
• Title: Efficient Large-Scale Cross-Domain Sequential Recommendation with Dynamic State Representations
• Authors: Manuel V. Loureiro, Steven Derby, Aleksei Medvedev, Alejandro Ariza-Casabona, Gonzalo Fiz Pontiveros, Tri Kurniawan Wijaya,
• Abstract summary: We introduce a novel approach for scalable multi-domain recommendation systems by replacing full inter-domain attention with two innovative mechanisms.<n>First, we propose novel positional embeddings that account for domain-transition specific information.<n>Second, we introduce a dynamic state representation for each domain, which is stored and accessed during subsequent token predictions.
• Score: 36.44078481424322
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, autoregressive recommendation models (ARMs), such as Meta's HSTU model, have emerged as a major breakthrough over traditional Deep Learning Recommendation Models (DLRMs), exhibiting the highly sought-after scaling law behaviour. However, when applied to multi-domain scenarios, the transformer architecture's attention maps become a computational bottleneck, as they attend to all items across every domain. To tackle this challenge, systems must efficiently balance inter and intra-domain knowledge transfer. In this work, we introduce a novel approach for scalable multi-domain recommendation systems by replacing full inter-domain attention with two innovative mechanisms: 1) Transition-Aware Positional Embeddings (TAPE): We propose novel positional embeddings that account for domain-transition specific information. This allows attention to be focused solely on intra-domain items, effectively reducing the unnecessary computational cost associated with attending to irrelevant domains. 2) Dynamic Domain State Representation (DDSR): We introduce a dynamic state representation for each domain, which is stored and accessed during subsequent token predictions. This enables the efficient transfer of relevant domain information without relying on full attention maps. Our method offers a scalable solution to the challenges posed by large-scale, multi-domain recommendation systems and demonstrates significant improvements in retrieval tasks by separately modelling and combining inter- and intra-domain representations.

Related papers

• Gaussian Mixture Flow Matching with Domain Alignment for Multi-Domain Sequential Recommendation [13.331414627413674]
  We propose textitGMFlowRec, an efficient generative framework for MDSR that models domain-aware transition trajectories.<n>Experiments on JD and Amazon datasets demonstrate that GMFlowRec achieves state-of-the-art performance with up to 44% improvement in NDCG@5.
  arXiv  Detail & Related papers  (2025-10-23T22:11:26Z)
• LLM-RecG: A Semantic Bias-Aware Framework for Zero-Shot Sequential Recommendation [5.512301280728178]
  Zero-shot cross-domain sequential recommendation (ZCDSR) enables predictions in unseen domains without additional training or fine-tuning.<n>Recent advancements in large language models (LLMs) have significantly enhanced ZCDSR by facilitating cross-domain knowledge transfer.<n>We propose a novel semantic bias-aware framework that improves cross-domain alignment at both the item and sequential levels.
  arXiv  Detail & Related papers  (2025-01-31T15:43:21Z)
• Adaptive Graph Integration for Cross-Domain Recommendation via Heterogeneous Graph Coordinators [31.05975545409408]
  Leveraging multi-domain data can improve recommendation systems by enriching user insights and mitigating data sparsity in individual domains.<n>We propose HAGO, a novel framework with textbfHeterogeneous textbfAdaptive textbfGraph cotextbfOrdinators.<n>Our framework adaptively adjusts the connections between coordinators and multi-domain graph nodes to enhance beneficial inter-domain interactions.
  arXiv  Detail & Related papers  (2024-10-15T15:50:53Z)
• Investigating the potential of Sparse Mixtures-of-Experts for multi-domain neural machine translation [59.41178047749177]
  We focus on multi-domain Neural Machine Translation, with the goal of developing efficient models which can handle data from various domains seen during training and are robust to domains unseen during training. We hypothesize that Sparse Mixture-of-Experts (SMoE) models are a good fit for this task, as they enable efficient model scaling. We conduct a series of experiments aimed at validating the utility of SMoE for the multi-domain scenario, and find that a straightforward width scaling of Transformer is a simpler and surprisingly more efficient approach in practice, and reaches the same performance level as SMoE.
  arXiv  Detail & Related papers  (2024-07-01T09:45:22Z)
• Large-Scale Multi-Domain Recommendation: an Automatic Domain Feature Extraction and Personalized Integration Framework [30.46152832695426]
  We propose an Automatic Domain Feature Extraction and Personalized Integration (DFEI) framework for the large-scale multi-domain recommendation. The framework automatically transforms the behavior of each individual user into an aggregation of all user behaviors within the domain, which serves as the domain features. Experimental results on both public and industrial datasets, consisting of over 20 domains, clearly demonstrate that the proposed framework achieves significantly better performance compared with SOTA baselines.
  arXiv  Detail & Related papers  (2024-04-12T09:57:17Z)
• Role Prompting Guided Domain Adaptation with General Capability Preserve for Large Language Models [55.51408151807268]
  When tailored to specific domains, Large Language Models (LLMs) tend to experience catastrophic forgetting. crafting a versatile model for multiple domains simultaneously often results in a decline in overall performance. We present the RolE Prompting Guided Multi-Domain Adaptation (REGA) strategy.
  arXiv  Detail & Related papers  (2024-03-05T08:22:41Z)
• Improving Intrusion Detection with Domain-Invariant Representation Learning in Latent Space [4.871119861180455]
  We introduce a two-phase representation learning technique using multi-task learning. We disentangle the latent space by minimizing the mutual information between the prior and latent space. We assess the model's efficacy across multiple cybersecurity datasets.
  arXiv  Detail & Related papers  (2023-12-28T17:24:13Z)
• Exploiting Graph Structured Cross-Domain Representation for Multi-Domain Recommendation [71.45854187886088]
  Multi-domain recommender systems benefit from cross-domain representation learning and positive knowledge transfer. We use temporal intra- and inter-domain interactions as contextual information for our method called MAGRec. We perform experiments on publicly available datasets in different scenarios where MAGRec consistently outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-02-12T19:51:32Z)
• TAL: Two-stream Adaptive Learning for Generalizable Person Re-identification [115.31432027711202]
  We argue that both domain-specific and domain-invariant features are crucial for improving the generalization ability of re-id models. We name two-stream adaptive learning (TAL) to simultaneously model these two kinds of information. Our framework can be applied to both single-source and multi-source domain generalization tasks.
  arXiv  Detail & Related papers  (2021-11-29T01:27:42Z)
• AFAN: Augmented Feature Alignment Network for Cross-Domain Object Detection [90.18752912204778]
  Unsupervised domain adaptation for object detection is a challenging problem with many real-world applications. We propose a novel augmented feature alignment network (AFAN) which integrates intermediate domain image generation and domain-adversarial training. Our approach significantly outperforms the state-of-the-art methods on standard benchmarks for both similar and dissimilar domain adaptations.
  arXiv  Detail & Related papers  (2021-06-10T05:01:20Z)
• Dual Attentive Sequential Learning for Cross-Domain Click-Through Rate Prediction [76.98616102965023]
  Cross domain recommender system constitutes a powerful method to tackle the cold-start and sparsity problem. We propose a novel approach to cross-domain sequential recommendations based on the dual learning mechanism.
  arXiv  Detail & Related papers  (2021-06-05T01:21:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.