WHALE: A Scalable Unified Model for Recommendation with Wukong-HSTU Architecture

• URL: http://arxiv.org/abs/2607.17017v1
• Date: Sun, 19 Jul 2026 00:59:09 GMT
• Title: WHALE: A Scalable Unified Model for Recommendation with Wukong-HSTU Architecture
• Abstract summary: We present WHALE, a scalable unified recommendation architecture that jointly models non-sequence and sequence features on top of Wukong and HSTU.<n>This design keeps both backbones active throughout the network and enables progressive Wukong-HSTU exchange.<n>On large-scale industrial recommendation data, WHALE achieves consistent gains in offline experiments.
• Score: 20.297096513302744
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As scalability becomes increasingly important in recommendation modeling, recent architectures have advanced the modeling of two broad sources of ranking signals along separate paths: non-sequence features, including user, item, context, and cross features; and sequence features from user behavior histories. Wukong and HSTU have emerged as representative scalable backbones for these paths: Wukong scales high-order non-sequence feature-interaction modeling, while HSTU scales long user-behavior sequence modeling. Despite their complementary strengths, practical architectures that combine these two types of feature modeling remain underexplored. We present WHALE, a scalable unified recommendation architecture that jointly models non-sequence and sequence features on top of Wukong and HSTU. Each WHALE layer contains a Wukong module, an HSTU module, and an attention-based fusion module in which Wukong-derived interaction representations query HSTU-derived behavior representations. This design keeps both backbones active throughout the network and enables progressive Wukong-HSTU exchange, allowing high-order feature crosses to repeatedly retrieve fine-grained evidence from long user histories. To make WHALE practical for industrial deployment, we introduce customized Triton kernels and other model-systems co-design techniques to improve training and inference efficiency. On large-scale industrial recommendation data, WHALE achieves consistent gains in offline experiments. Additionally, it delivers positive online gains with a modest serving-throughput trade-off. The method has been deployed in production systems. Overall, WHALE provides a practical example of how these two sources of information can be scalably unified in an industrial recommendation model.

Related papers

• UniFormer: Efficient and Unified Model-Centric Scaling for Industrial Recommendation [46.05454633225205]
  UniFormer is an efficient and unified model-centric scaling framework for industrial recommender systems.<n>To improve efficiency, UniFormer decomposes the overall modeling space into feature and task spaces.<n>UniFormer consistently improves user engagement and interaction metrics.
  arXiv  Detail & Related papers  (2026-06-25T14:03:28Z)
• TokenFormer: Unify the Multi-Field and Sequential Recommendation Worlds [17.505701673051405]
  We propose TokenFormer, a unified recommendation architecture with the following innovations.<n>First, we introduce a Bottom-Full-Top-Sliding (BFTS) attention scheme, which applies full self-attention in the lower layers and shrinking-window sliding attention in the upper layers.<n>Second, we introduce a Non-linear Interaction Representation (NLIR) that applies one-sided non-linear multiplicative transformations to the hidden states.
  arXiv  Detail & Related papers  (2026-04-15T11:25:46Z)
• IAT: Instance-As-Token Compression for Historical User Sequence Modeling in Industrial Recommender Systems [19.111182803412458]
  Instance-As-Token (IAT) is a novel two-stage sequence modeling framework.<n>IAT encodes historical interaction patterns in a compact yet informative token.<n>IAT has been successfully deployed in real-world industrial recommender systems.
  arXiv  Detail & Related papers  (2026-04-10T04:02:58Z)
• MixFormer: Co-Scaling Up Dense and Sequence in Industrial Recommenders [11.566232697512879]
  MixFormer is a unified Transformer-style architecture tailored for recommender systems.<n>It jointly models sequential behaviors and feature interactions within a single backbone.<n>Experiments on large-scale industrial datasets demonstrate that MixFormer consistently exhibits superior accuracy and efficiency.
  arXiv  Detail & Related papers  (2026-02-15T11:53:30Z)
• HyFormer: Revisiting the Roles of Sequence Modeling and Feature Interaction in CTR Prediction [8.97787361529607]
  This paper presents HyFormer, a unified hybrid transformer architecture that tightly integrates long-sequence modeling and feature interaction into a single backbone.<n>Experiments on billion-scale industrial datasets demonstrate that HyFormer consistently outperforms strong LONGER and RankMixer baselines.
  arXiv  Detail & Related papers  (2026-01-19T02:55:05Z)
• FuXi-$α$: Scaling Recommendation Model with Feature Interaction Enhanced Transformer [81.12174905444229]
  Recent advancements have shown that expanding sequential recommendation models to large-scale recommendation models can be an effective strategy.<n>We propose a new model called FuXi-$alpha$ to address these issues.<n>Our model outperforms existing models, with its performance continuously improving as the model size increases.
  arXiv  Detail & Related papers  (2025-02-05T09:46:54Z)
• SIGMA: Selective Gated Mamba for Sequential Recommendation [56.85338055215429]
  Mamba, a recent advancement, has exhibited exceptional performance in time series prediction.<n>We introduce a new framework named Selective Gated Mamba ( SIGMA) for Sequential Recommendation.<n>Our results indicate that SIGMA outperforms current models on five real-world datasets.
  arXiv  Detail & Related papers  (2024-08-21T09:12:59Z)
• Noise-powered Multi-modal Knowledge Graph Representation Framework [52.95468915728721]
  The rise of Multi-modal Pre-training highlights the necessity for a unified Multi-Modal Knowledge Graph representation learning framework.<n>We propose a novel SNAG method that utilizes a Transformer-based architecture equipped with modality-level noise masking.<n>Our approach achieves SOTA performance across a total of ten datasets, demonstrating its versatility.
  arXiv  Detail & Related papers  (2024-03-11T15:48:43Z)
• Multi-Scale Semantics-Guided Neural Networks for Efficient Skeleton-Based Human Action Recognition [140.18376685167857]
  A simple yet effective multi-scale semantics-guided neural network is proposed for skeleton-based action recognition. MS-SGN achieves the state-of-the-art performance on the NTU60, NTU120, and SYSU datasets.
  arXiv  Detail & Related papers  (2021-11-07T03:50:50Z)
• Polynomial Networks in Deep Classifiers [55.90321402256631]
  We cast the study of deep neural networks under a unifying framework. Our framework provides insights on the inductive biases of each model. The efficacy of the proposed models is evaluated on standard image and audio classification benchmarks.
  arXiv  Detail & Related papers  (2021-04-16T06:41:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.