Preference-Drift-Aware Subsequence Learning and Hierarchical Context Fusion for Long-Sequence Generative Recommendation

• URL: http://arxiv.org/abs/2609.12556v1
• Date: Fri, 11 Sep 2026 08:01:29 GMT
• Title: Preference-Drift-Aware Subsequence Learning and Hierarchical Context Fusion for Long-Sequence Generative Recommendation
• Abstract summary: Long-sequence generative recommendation methods autoregressively model the user's interaction sequence to generate the next-item representation.<n>Existing methods fall into two categories: efficient full-sequence modeling and target-aware context retrieval.<n>We propose a preference-drift-aware subsequence learning and hierarchical context fusion for long-sequence generative recommendation.
• Score: 20.33597628832828
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Long-sequence generative recommendation methods autoregressively model the user's interaction sequence to generate the next-item representation. Existing methods generally fall into two categories: efficient full-sequence modeling and target-aware context retrieval. Our experiments reveal that as the sequence length increases, the former incurs steadily growing computational cost while its accuracy gains quickly saturate and even degrade due to noise; the latter, though shortening the input sequence, is susceptible to noise that is semantically consistent yet preference-inconsistent, as well as to incomplete contexts. Both paradigms ignore the dynamic changes of user preferences and the cross-subsequence dependencies when handling historical information, thereby limiting accuracy and efficiency. To address these issues, we propose a preference-drift-aware subsequence learning and hierarchical context fusion for long-sequence generative recommendation. Specifically, we learn differentiable soft subsequence boundaries using multidimensional preference-drift information and aggregate items within each subsequence into preference-coherent representations via linear attention with soft assignment weights, thereby circumventing the expense of full-sequence attention. A cross-attention mechanism is then employed to capture dependencies between recent interactions and relevant subsequence contexts, mitigating noise in learning recent-item representations. Finally, a gated fusion mechanism adaptively combines the recent-item representation with the global subsequence context, allowing the resulting target representation to encode both recent and long-term preferences. Extensive experiments demonstrate that our method consistently outperforms existing baselines in both recommendation accuracy and computational efficiency.

Related papers

• Rethinking Convolutional Networks for Attribute-Aware Sequential Recommendation [6.846413131554734]
  Attribute-aware sequential recommendation entails predicting the next item a user will interact with based on a chronologically ordered history of past interactions.<n>Existing methods typically leverage self-attention mechanisms to aggregate the entire sequence into a unified representation.<n>We propose ConvRec, which employs convolutional layers in a hierarchical, down-scaled fashion to generate compact, yet expressive sequence representations.
  arXiv  Detail & Related papers  (2026-05-06T10:18:43Z)
• Mixture of Sequence: Theme-Aware Mixture-of-Experts for Long-Sequence Recommendation [48.98369214412951]
  We propose a model-agnostic MoE approach that achieves accurate predictions by extracting theme-specific and multi-scale subsequences from noisy raw user sequences.<n> Experimental results demonstrate that MoS consistently achieves the SOTA performance while introducing fewer FLOPs.
  arXiv  Detail & Related papers  (2026-03-01T23:20:48Z)
• GEMs: Breaking the Long-Sequence Barrier in Generative Recommendation with a Multi-Stream Decoder [54.64137490632567]
  We propose a novel and unified framework designed to capture users' sequences from long-term history.<n>Generative Multi-streamers ( GEMs) break user sequences into three streams.<n>Extensive experiments on large-scale industrial datasets demonstrate that GEMs significantly outperforms state-the-art methods in recommendation accuracy.
  arXiv  Detail & Related papers  (2026-02-14T06:42:56Z)
• Multi-granularity Interest Retrieval and Refinement Network for Long-Term User Behavior Modeling in CTR Prediction [68.90783662117936]
  Click-through Rate (CTR) prediction is crucial for online personalization platforms.<n>Recent advancements have shown that modeling rich user behaviors can significantly improve the performance of CTR prediction.<n>We propose Multi-granularity Interest Retrieval and Refinement Network (MIRRN)
  arXiv  Detail & Related papers  (2024-11-22T15:29:05Z)
• Long-Sequence Recommendation Models Need Decoupled Embeddings [49.410906935283585]
  We identify and characterize a neglected deficiency in existing long-sequence recommendation models.<n>A single set of embeddings struggles with learning both attention and representation, leading to interference between these two processes.<n>We propose the Decoupled Attention and Representation Embeddings (DARE) model, where two distinct embedding tables are learned separately to fully decouple attention and representation.
  arXiv  Detail & Related papers  (2024-10-03T15:45:15Z)
• IFA: Interaction Fidelity Attention for Entire Lifelong Behaviour Sequence Modeling [25.951109597584747]
  The lifelong user behavior sequence provides abundant information of user preference and gains impressive improvement in the recommendation task. To meet the severe latency requirement in online service, a short sub-sequence is sampled based on similarity to the target item. We propose a new efficient paradigm to model the full lifelong sequence, which is named as textbfInteraction textbfFidelity textbfAttention (textbfIFA)
  arXiv  Detail & Related papers  (2024-06-14T06:16:03Z)
• Diffusion-based Contrastive Learning for Sequential Recommendation [6.3482831836623355]
  We propose a Context-aware Diffusion-based Contrastive Learning for Sequential Recommendation, named CaDiRec.<n>CaDiRec employs a context-aware diffusion model to generate alternative items for the given positions within a sequence.<n>We train the entire framework in an end-to-end manner, with shared item embeddings between the diffusion model and the recommendation model.
  arXiv  Detail & Related papers  (2024-05-15T14:20:37Z)
• Learning Sequence Representations by Non-local Recurrent Neural Memory [61.65105481899744]
  We propose a Non-local Recurrent Neural Memory (NRNM) for supervised sequence representation learning. Our model is able to capture long-range dependencies and latent high-level features can be distilled by our model. Our model compares favorably against other state-of-the-art methods specifically designed for each of these sequence applications.
  arXiv  Detail & Related papers  (2022-07-20T07:26:15Z)
• Learning Temporal Point Processes for Efficient Retrieval of Continuous Time Event Sequences [24.963828650935913]
  We propose NEUROSEQRET which learns to retrieve and rank a relevant set of continuous-time event sequences for a given query sequence. We develop two variants of the relevance model which offer a tradeoff between accuracy and efficiency. Our experiments with several datasets show the significant accuracy boost of NEUROSEQRET beyond several baselines.
  arXiv  Detail & Related papers  (2022-02-17T11:16:31Z)
• Contrastive Self-supervised Sequential Recommendation with Robust Augmentation [101.25762166231904]
  Sequential Recommendationdescribes a set of techniques to model dynamic user behavior in order to predict future interactions in sequential user data. Old and new issues remain, including data-sparsity and noisy data. We propose Contrastive Self-Supervised Learning for sequential Recommendation (CoSeRec)
  arXiv  Detail & Related papers  (2021-08-14T07:15:25Z)
• Sequence Adaptation via Reinforcement Learning in Recommender Systems [8.909115457491522]
  We propose the SAR model, which learns the sequential patterns and adjusts the sequence length of user-item interactions in a personalized manner. In addition, we optimize a joint loss function to align the accuracy of the sequential recommendations with the expected cumulative rewards of the critic network. Our experimental evaluation on four real-world datasets demonstrates the superiority of our proposed model over several baseline approaches.
  arXiv  Detail & Related papers  (2021-07-31T13:56:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.