ChronicleRec: Pre-training Temporally Anchored Tokens for Lifelong User Modeling

• URL: http://arxiv.org/abs/2609.12375v1
• Date: Fri, 11 Sep 2026 02:47:55 GMT
• Title: ChronicleRec: Pre-training Temporally Anchored Tokens for Lifelong User Modeling
• Abstract summary: Existing lifelong-interest methods retrieve target-relevant behaviors for each candidate.<n>Recent target-independent compression methods enable cached user summaries, but often append query tokens at the sequence end.<n>We propose ChronicleRec, a pre-train-and-transfer framework that compresses an ultra-long behavior sequence once into a chronologically ordered set of Chronicle Tokens.
• Score: 19.519280627157716
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Modeling ultra-long user behavior sequences is crucial for industrial recommendation and online advertising, yet directly feeding thousands of historical actions into ranking models is computationally prohibitive, while truncation discards long-range signals. Existing lifelong-interest methods retrieve target-relevant behaviors for each candidate, coupling long-sequence modeling with candidate scoring and repeated online cost. Recent target-independent compression methods enable cached user summaries, but often append query tokens at the sequence end and use bidirectional encoding, producing unordered and redundant summaries that overlook temporal structure. We propose ChronicleRec, a pre-train-and-transfer framework that compresses an ultra-long behavior sequence once into a chronologically ordered set of Chronicle Tokens. ChronicleRec applies a recency-aware multi-granularity merge, preserving recent behaviors while coarsening distant history. It then interleaves query tokens with the merged sequence and uses a causal encoder, so each query summarizes only the history before its temporal anchor. A multi-horizon design masks different recent-history windows across parallel branches to learn complementary long-range interests. The compressor is pre-trained with a mask-and-predict objective that reconstructs held-out recent behaviors from compressed older history, aligning historical signals with near-present intent. Since Chronicle Tokens are target-independent, they can be cached per user, decoupling ultra-long sequence modeling from online candidate scoring. Experiments on KuaiRand and Tencent AdLive show that ChronicleRec outperforms recent-window and single-pass compression baselines while approaching full-attention performance. Token analyses reveal temporally organized and complementary representations, and a seven-day online A/B test confirms significant production gains.

Related papers

• Preference-Drift-Aware Subsequence Learning and Hierarchical Context Fusion for Long-Sequence Generative Recommendation [20.33597628832828]
  Long-sequence generative recommendation methods autoregressively model the user's interaction sequence to generate the next-item representation.<n>Existing methods fall into two categories: efficient full-sequence modeling and target-aware context retrieval.<n>We propose a preference-drift-aware subsequence learning and hierarchical context fusion for long-sequence generative recommendation.
  arXiv  Detail & Related papers  (2026-09-11T08:01:29Z)
• SequenceO1: End-to-End Ultra-Long (100K) Sequence Modeling in Recommendation with Low-Rank Caching [17.335538522124555]
  SequenceO1 is an end-to-end framework for ultra-long user behavior sequence modeling.<n>It is deployed at full traffic on Douyin with histories of up to 100K interactions.<n>Results show consistent offline and online gains, while sketch retains most of the benefit of directly scaling end-to-end sequence ranking to 100K.
  arXiv  Detail & Related papers  (2026-09-08T08:49:44Z)
• Rethinking Convolutional Networks for Attribute-Aware Sequential Recommendation [6.846413131554734]
  Attribute-aware sequential recommendation entails predicting the next item a user will interact with based on a chronologically ordered history of past interactions.<n>Existing methods typically leverage self-attention mechanisms to aggregate the entire sequence into a unified representation.<n>We propose ConvRec, which employs convolutional layers in a hierarchical, down-scaled fashion to generate compact, yet expressive sequence representations.
  arXiv  Detail & Related papers  (2026-05-06T10:18:43Z)
• Recurrent Preference Memory for Efficient Long-Sequence Generative Recommendation [27.325586037888]
  We introduce Rec2PM, a framework that compresses long user interaction histories into compact Preference Memory tokens.<n>Experiments show that Rec2PM significantly reduces inference latency and memory footprint while achieving superior accuracy compared to full-sequence models.
  arXiv  Detail & Related papers  (2026-02-12T05:51:52Z)
• CoFiRec: Coarse-to-Fine Tokenization for Generative Recommendation [55.783414010717074]
  CoFiRec is a novel generative recommendation framework that decomposes item information into semantic levels.<n>We show that CoFiRec outperforms existing methods, offering a new perspective for generative recommendation.
  arXiv  Detail & Related papers  (2025-11-27T18:59:35Z)
• Deep Multiple Quantization Network on Long Behavior Sequence for Click-Through Rate Prediction [5.465812199325144]
  In Click-Through Rate (CTR) prediction, the long behavior sequence has a vital influence on assessing the user's interest in the candidate item.<n>We propose the Deep Multiple Quantization Network (DMQN) to process long behavior sequence end-to-end through compressing the long behavior sequence.<n>The A/B test in our advertising system shows that DMQN improves CTR by 3.5% and RPM by 2.0%.
  arXiv  Detail & Related papers  (2025-08-28T14:58:47Z)
• RefreshKV: Updating Small KV Cache During Long-form Generation [54.00118604124301]
  We propose a new inference method, RefreshKV, that flexibly alternates between full context attention and attention over a subset of input tokens during generation.<n>Applying our method to off-the-shelf LLMs achieves comparable speedup to eviction-based methods while improving performance for various long-form generation tasks.
  arXiv  Detail & Related papers  (2024-11-08T18:57:07Z)
• Parallel Decoding via Hidden Transfer for Lossless Large Language Model Acceleration [54.897493351694195]
  We propose a novel parallel decoding approach, namely textithidden transfer, which decodes multiple successive tokens simultaneously in a single forward pass. In terms of acceleration metrics, we outperform all the single-model acceleration techniques, including Medusa and Self-Speculative decoding.
  arXiv  Detail & Related papers  (2024-04-18T09:17:06Z)
• Contrastive Self-supervised Sequential Recommendation with Robust Augmentation [101.25762166231904]
  Sequential Recommendationdescribes a set of techniques to model dynamic user behavior in order to predict future interactions in sequential user data. Old and new issues remain, including data-sparsity and noisy data. We propose Contrastive Self-Supervised Learning for sequential Recommendation (CoSeRec)
  arXiv  Detail & Related papers  (2021-08-14T07:15:25Z)
• Augmenting Sequential Recommendation with Pseudo-Prior Items via Reversely Pre-training Transformer [61.818320703583126]
  Sequential Recommendation characterizes the evolving patterns by modeling item sequences chronologically. Recent developments of transformer inspire the community to design effective sequence encoders. We introduce a new framework for textbfAugmenting textbfSequential textbfRecommendation with textbfPseudo-prior items(ASReP)
  arXiv  Detail & Related papers  (2021-05-02T18:06:23Z)
• Dynamic Memory based Attention Network for Sequential Recommendation [79.5901228623551]
  We propose a novel long sequential recommendation model called Dynamic Memory-based Attention Network (DMAN) It segments the overall long behavior sequence into a series of sub-sequences, then trains the model and maintains a set of memory blocks to preserve long-term interests of users. Based on the dynamic memory, the user's short-term and long-term interests can be explicitly extracted and combined for efficient joint recommendation.
  arXiv  Detail & Related papers  (2021-02-18T11:08:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.