IAT: Instance-As-Token Compression for Historical User Sequence Modeling in Industrial Recommender Systems

• URL: http://arxiv.org/abs/2604.08933v1
• Date: Fri, 10 Apr 2026 04:02:58 GMT
• Title: IAT: Instance-As-Token Compression for Historical User Sequence Modeling in Industrial Recommender Systems
• Abstract summary: Instance-As-Token (IAT) is a novel two-stage sequence modeling framework.<n>IAT encodes historical interaction patterns in a compact yet informative token.<n>IAT has been successfully deployed in real-world industrial recommender systems.
• Score: 19.111182803412458
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Although sophisticated sequence modeling paradigms have achieved remarkable success in recommender systems, the information capacity of hand-crafted sequential features constrains the performance upper bound. To better enhance user experience by encoding historical interaction patterns, this paper presents a novel two-stage sequence modeling framework termed Instance-As-Token (IAT). The first stage of IAT compresses all features of each historical interaction instance into a unified instance embedding, which encodes the interaction characteristics in a compact yet informative token. Both temporal-order and user-order compression schemes are proposed, with the latter better aligning with the demands of downstream sequence modeling. The second stage involves the downstream task fetching fixed-length compressed instance tokens via timestamps and adopting standard sequence modeling approaches to learn long-range preferences patterns. Extensive experiments demonstrate that IAT significantly outperforms state-of-the-art methods and exhibits superior in-domain and cross-domain transferability. IAT has been successfully deployed in real-world industrial recommender systems, including e-commerce advertising, shopping mall marketing, and live-streaming e-commerce, delivering substantial improvements in key business metrics.

Related papers

• SITA: Semantic Interest Tokens for Target-Aware Compression in Long-Sequence Recommendation [69.02976266748193]
  We propose textbfSITA, a target-aware compression framework for long-sequence recommendation.<n>SITA adaptively aggregates the corresponding structured interests to construct the target-specific user representation.<n>Experiments on public datasets and a large-scale industrial dataset demonstrate that SITA consistently outperforms representative baselines.
  arXiv  Detail & Related papers  (2026-08-04T13:59:44Z)
• Rethinking Convolutional Networks for Attribute-Aware Sequential Recommendation [6.846413131554734]
  Attribute-aware sequential recommendation entails predicting the next item a user will interact with based on a chronologically ordered history of past interactions.<n>Existing methods typically leverage self-attention mechanisms to aggregate the entire sequence into a unified representation.<n>We propose ConvRec, which employs convolutional layers in a hierarchical, down-scaled fashion to generate compact, yet expressive sequence representations.
  arXiv  Detail & Related papers  (2026-05-06T10:18:43Z)
• Action-Aware Generative Sequence Modeling for Short Video Recommendation [50.74467504063892]
  Short videos often contain diverse segments, and users may not hold the same attitude toward all of them.<n>Traditional binary-classification recommendation models, which treat a video as a single holistic entity, face limitations in accurately capturing such nuanced preferences.<n>We propose a novel modeling paradigm: Action-Aware Generative Sequence Network (A2Gen), which refines user actions along the temporal dimension.
  arXiv  Detail & Related papers  (2026-04-28T16:41:04Z)
• TokenFormer: Unify the Multi-Field and Sequential Recommendation Worlds [17.505701673051405]
  We propose TokenFormer, a unified recommendation architecture with the following innovations.<n>First, we introduce a Bottom-Full-Top-Sliding (BFTS) attention scheme, which applies full self-attention in the lower layers and shrinking-window sliding attention in the upper layers.<n>Second, we introduce a Non-linear Interaction Representation (NLIR) that applies one-sided non-linear multiplicative transformations to the hidden states.
  arXiv  Detail & Related papers  (2026-04-15T11:25:46Z)
• Query as Anchor: Scenario-Adaptive User Representation via Large Language Model [28.30329175937291]
  We propose Query-as-Anchor, a framework shifting user modeling from static encoding to dynamic, query-aware synthesis.<n>We first construct UserU, an industrial-scale pre-training dataset that aligns behavioral sequences with user understanding semantics.<n>We introduce Cluster-based Soft Prompt Tuning to enforce discriminative latent structures.<n>For deployment, anchoring queries at sequence termini enables KV-cache-accelerated inference with negligible incremental latency.
  arXiv  Detail & Related papers  (2026-02-16T06:09:31Z)
• MixFormer: Co-Scaling Up Dense and Sequence in Industrial Recommenders [11.566232697512879]
  MixFormer is a unified Transformer-style architecture tailored for recommender systems.<n>It jointly models sequential behaviors and feature interactions within a single backbone.<n>Experiments on large-scale industrial datasets demonstrate that MixFormer consistently exhibits superior accuracy and efficiency.
  arXiv  Detail & Related papers  (2026-02-15T11:53:30Z)
• HyFormer: Revisiting the Roles of Sequence Modeling and Feature Interaction in CTR Prediction [8.97787361529607]
  This paper presents HyFormer, a unified hybrid transformer architecture that tightly integrates long-sequence modeling and feature interaction into a single backbone.<n>Experiments on billion-scale industrial datasets demonstrate that HyFormer consistently outperforms strong LONGER and RankMixer baselines.
  arXiv  Detail & Related papers  (2026-01-19T02:55:05Z)
• STAR-Rec: Making Peace with Length Variance and Pattern Diversity in Sequential Recommendation [61.320991769685065]
  STAR-Rec is a novel architecture that combines preference-aware attention and state-space modeling.<n>We show that STAR-Rec consistently outperforms state-of-the-art sequential recommendation methods.
  arXiv  Detail & Related papers  (2025-05-06T12:40:38Z)
• SIGMA: Selective Gated Mamba for Sequential Recommendation [56.85338055215429]
  Mamba, a recent advancement, has exhibited exceptional performance in time series prediction.<n>We introduce a new framework named Selective Gated Mamba ( SIGMA) for Sequential Recommendation.<n>Our results indicate that SIGMA outperforms current models on five real-world datasets.
  arXiv  Detail & Related papers  (2024-08-21T09:12:59Z)
• Contrastive Self-supervised Sequential Recommendation with Robust Augmentation [101.25762166231904]
  Sequential Recommendationdescribes a set of techniques to model dynamic user behavior in order to predict future interactions in sequential user data. Old and new issues remain, including data-sparsity and noisy data. We propose Contrastive Self-Supervised Learning for sequential Recommendation (CoSeRec)
  arXiv  Detail & Related papers  (2021-08-14T07:15:25Z)
• Dual Attentive Sequential Learning for Cross-Domain Click-Through Rate Prediction [76.98616102965023]
  Cross domain recommender system constitutes a powerful method to tackle the cold-start and sparsity problem. We propose a novel approach to cross-domain sequential recommendations based on the dual learning mechanism.
  arXiv  Detail & Related papers  (2021-06-05T01:21:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.