RoleMix: Unifying Sequential and Non-Sequential Features via Semantic Tokenization for Post-Click Conversion Rate Prediction

• URL: http://arxiv.org/abs/2607.22700v1
• Date: Sat, 18 Jul 2026 15:44:48 GMT
• Title: RoleMix: Unifying Sequential and Non-Sequential Features via Semantic Tokenization for Post-Click Conversion Rate Prediction
• Abstract summary: RoleMix represents sequential and non-sequential evidence through a shared, role-preserving token interface.<n>On the large-scale KDD Cup 2026 Tencent UniRec Challenge, RoleMix achieves 83.648% online AUC, outperforming the official industrial baseline by 1.953%.
• Score: 1.6050320174037502
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Post-click conversion rate (PCVR) prediction is central to industrial recommendation, but remains challenged by the structural mismatch between sparse, unordered multi-field features and long, domain-specific behavior histories. Existing models often process these signals through separate pathways and fuse them late, weakening semantic roles and limiting cross-signal refinement. We propose RoleMix, a unified interaction architecture that represents sequential and non-sequential evidence through a shared, role-preserving token interface. Non-sequential fields are converted into explicit semantic tokens that preserve user, item, pairwise, dense, contextual, and cross-feature roles, while long behavior domains are compressed into item- and context-aware sequence-query tokens through two-stage hierarchical window attention. The resulting global, semantic, and sequence-query tokens are jointly refined by stacked UniMixing-Lite blocks for PCVR prediction. On the large-scale KDD Cup 2026 Tencent UniRec Challenge, RoleMix achieves 83.648% online AUC, outperforming the official industrial baseline by 1.953%. Ablation studies show that semantic tokenization yields the largest isolated gain, highlighting a key principle for large-scale PCVR modeling: preserving field semantics at the token-interface level is as important as scaling the interaction backbone.

Related papers

• Accelerating Unified Multimodal Models with Core-Expansion Routing and Unified Computation Scheduling [61.28599393707943]
  Unified multimodal models jointly support understanding and generation, but incur substantial redundant computation across tokens, layers, and generation timesteps.<n>We propose a task-shared core scorer and progress-conditioned generation expansions, optimized through generation decomposition and cross-task core alignment.<n> Experiments on two representative UMM architectures demonstrate consistent quality--efficiency improvements across both tasks, retaining 98.03% of dense understanding performance with a 1.93times$ end-to-end inference speedup.
  arXiv  Detail & Related papers  (2026-08-29T14:27:55Z)
• Topology-Masked Unified Backbone for Joint Feature Interaction and Multi-Domain Sequence Modeling [8.950162084237288]
  MaskRec is a topology-masked unified token interaction architecture for feature interaction and multi-domain sequence modeling.<n>Based on this unified token space, MaskRec designs a structured attention mask, TopoMask, which selectively enables or blocks attention connections.<n>MaskRec achieves stable improvements over the official baseline, validating the effectiveness of the proposed unified framework for industrial CVR prediction.
  arXiv  Detail & Related papers  (2026-08-27T11:49:55Z)
• UniDot: A Unified Network for Sequence Modeling and Feature Interaction in Large-scale Recommendation [3.59042069607434]
  We present UniDot, a novel architecture for post-click conversion prediction built from the factorization-machine point of view.<n>A single dot-product of tokens can underlie both feature interaction and sequence modeling.<n>UniDot finished as the runner-up on the Industrial track of the TAAC KDD Cup 2026.
  arXiv  Detail & Related papers  (2026-08-17T16:52:56Z)
• STAR: Structured Tokenization and Target-Aware Interest Representation for PCVR Prediction [7.574291100593417]
  We present STAR (Structured Tokenization and Target-Aware Interest Representation), a framework for the KDD Cup 2026 Tencent UniRec Challenge.<n> STAR combines structured feature tokenization with target-aware interest representation on top of a HyFormer-style multi-sequence backbone.<n> Experiments on the challenge dataset identify the components that most reliably improve ranking AUC, while LogLoss is reported as a calibration diagnostic.
  arXiv  Detail & Related papers  (2026-08-13T09:09:54Z)
• SITA: Semantic Interest Tokens for Target-Aware Compression in Long-Sequence Recommendation [69.02976266748193]
  We propose textbfSITA, a target-aware compression framework for long-sequence recommendation.<n>SITA adaptively aggregates the corresponding structured interests to construct the target-specific user representation.<n>Experiments on public datasets and a large-scale industrial dataset demonstrate that SITA consistently outperforms representative baselines.
  arXiv  Detail & Related papers  (2026-08-04T13:59:44Z)
• Drift-Aware Continual Tokenization for Generative Recommendation [44.13013639783267]
  Generative recommendation commonly adopts a two-stage pipeline in which a learnable tokenizer maps items to discrete token sequences.<n>Recent tokenizers further incorporate collaborative signals so that items with similar user-behavior patterns receive similar codes.<n>New items cause collision and shifts, while new interactions induce collaborative drift in existing items.<n>We propose DACT, a Drift-Aware Continual Tokenization framework with two stages: tokenizer fine-tuning and item-level drift confidence.
  arXiv  Detail & Related papers  (2026-03-31T13:02:47Z)
• HyFormer: Revisiting the Roles of Sequence Modeling and Feature Interaction in CTR Prediction [8.97787361529607]
  This paper presents HyFormer, a unified hybrid transformer architecture that tightly integrates long-sequence modeling and feature interaction into a single backbone.<n>Experiments on billion-scale industrial datasets demonstrate that HyFormer consistently outperforms strong LONGER and RankMixer baselines.
  arXiv  Detail & Related papers  (2026-01-19T02:55:05Z)
• MergeDNA: Context-aware Genome Modeling with Dynamic Tokenization through Token Merging [65.07273789940116]
  This paper introduces a hierarchical architecture that jointly optimize a dynamic genomic tokenizer and latent Transformers with context-aware pre-training tasks.<n> MergeDNA achieves superior performance on three popular DNA benchmarks and several multi-omics tasks with fine-tuning or zero-shot evaluation.
  arXiv  Detail & Related papers  (2025-11-17T19:27:41Z)
• Practice on Long Behavior Sequence Modeling in Tencent Advertising [75.65309022911994]
  Long-sequence modeling has become an indispensable frontier in recommendation systems for capturing users' long-term preferences.<n>We propose several practical approaches within the two-stage framework for long-sequence modeling.<n> Deployed in production on Tencent's large-scale advertising platforms, our innovations delivered significant performance gains.
  arXiv  Detail & Related papers  (2025-09-10T06:55:57Z)
• Beyond BEV: Optimizing Point-Level Tokens for Collaborative Perception [17.654858416126093]
  Collaborative perception allows agents to enhance their perceptual capabilities by exchanging intermediate features.<n>Existing methods typically organize these intermediate features as 2D bird's-eye-view (BEV) representations.<n>We present CoPLOT, a novel Collaborative perception framework that utilizes Point-Level Optimized Tokens.
  arXiv  Detail & Related papers  (2025-08-27T07:27:42Z)
• CTR-Sink: Attention Sink for Language Models in Click-Through Rate Prediction [42.92011330807996]
  $textitCTR-Sink$ is a novel framework introducing behavior-level attention sinks tailored for recommendation scenarios.<n>Inspired by attention sink theory, it constructs attention focus sinks and dynamically regulates attention aggregation via external information.
  arXiv  Detail & Related papers  (2025-08-05T17:30:34Z)
• Multi-granularity Interest Retrieval and Refinement Network for Long-Term User Behavior Modeling in CTR Prediction [68.90783662117936]
  Click-through Rate (CTR) prediction is crucial for online personalization platforms.<n>Recent advancements have shown that modeling rich user behaviors can significantly improve the performance of CTR prediction.<n>We propose Multi-granularity Interest Retrieval and Refinement Network (MIRRN)
  arXiv  Detail & Related papers  (2024-11-22T15:29:05Z)
• Object Segmentation by Mining Cross-Modal Semantics [68.88086621181628]
  We propose a novel approach by mining the Cross-Modal Semantics to guide the fusion and decoding of multimodal features. Specifically, we propose a novel network, termed XMSNet, consisting of (1) all-round attentive fusion (AF), (2) coarse-to-fine decoder (CFD), and (3) cross-layer self-supervision.
  arXiv  Detail & Related papers  (2023-05-17T14:30:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.