Trie-Aware Transformers for Generative Recommendation

• URL: http://arxiv.org/abs/2602.21677v1
• Date: Wed, 25 Feb 2026 08:25:16 GMT
• Title: Trie-Aware Transformers for Generative Recommendation
• Authors: Zhenxiang Xu, Jiawei Chen, Sirui Chen, Yong He, Jieyu Yang, Chuan Yuan, Ke Ding, Can Wang,
• Abstract summary: We propose TrieRec, a trie-aware generative recommendation method that augments Transformers with structural inductive biases.<n>We implement TrieRec within three representative GR backbones, achieving notably improvements of 8.83% on average across four real-world datasets.
• Score: 18.901863061905825
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generative recommendation (GR) aligns with advances in generative AI by casting next-item prediction as token-level generation rather than score-based ranking. Most GR methods adopt a two-stage pipeline: (i) \textit{item tokenization}, which maps each item to a sequence of discrete, hierarchically organized tokens; and (ii) \textit{autoregressive generation}, which predicts the next item's tokens conditioned on the tokens of user's interaction history. Although hierarchical tokenization induces a prefix tree (trie) over items, standard autoregressive modeling with conventional Transformers often flattens item tokens into a linear stream and overlooks the underlying topology. To address this, we propose TrieRec, a trie-aware generative recommendation method that augments Transformers with structural inductive biases via two positional encodings. First, a \textit{trie-aware absolute positional encoding} aggregates a token's (node's) local structural context (\eg depth, ancestors, and descendants) into the token representation. Second, a \textit{topology-aware relative positional encoding} injects pairwise structural relations into self-attention to capture topology-induced semantic relatedness. TrieRec is also model-agnostic, efficient, and hyperparameter-free. In our experiments, we implement TrieRec within three representative GR backbones, achieving notably improvements of 8.83\% on average across four real-world datasets.

Related papers

• AutoRegressive Generation with B-rep Holistic Token Sequence Representation [31.0473553479822]
  We propose BrepARG, the first attempt to encode B-rep's geometry and topology into a holistic token sequence representation.<n>Specifically, BrepARG encodes B-rep into 3 types of tokens: geometry and position tokens representing geometric features, and face index tokens representing topology.<n> Experiments demonstrate that BrepARG achieves state-of-the-art (SOTA) performance.
  arXiv  Detail & Related papers  (2026-01-23T14:15:02Z)
• CoFiRec: Coarse-to-Fine Tokenization for Generative Recommendation [55.783414010717074]
  CoFiRec is a novel generative recommendation framework that decomposes item information into semantic levels.<n>We show that CoFiRec outperforms existing methods, offering a new perspective for generative recommendation.
  arXiv  Detail & Related papers  (2025-11-27T18:59:35Z)
• BrepGPT: Autoregressive B-rep Generation with Voronoi Half-Patch [61.20046418942948]
  Boundary representation (B-rep) is the de facto standard for CAD model representation in modern industrial design.<n>We present BrepGPT, a single-stage autoregressive framework for B-rep generation.
  arXiv  Detail & Related papers  (2025-11-27T07:16:53Z)
• MergeDNA: Context-aware Genome Modeling with Dynamic Tokenization through Token Merging [65.07273789940116]
  This paper introduces a hierarchical architecture that jointly optimize a dynamic genomic tokenizer and latent Transformers with context-aware pre-training tasks.<n> MergeDNA achieves superior performance on three popular DNA benchmarks and several multi-omics tasks with fine-tuning or zero-shot evaluation.
  arXiv  Detail & Related papers  (2025-11-17T19:27:41Z)
• LLaDA-Rec: Discrete Diffusion for Parallel Semantic ID Generation in Generative Recommendation [32.284624021041004]
  We propose LLaDA-Rec, a discrete diffusion framework that reformulates recommendation as parallel semantic ID generation.<n> Experiments on three real-world datasets show that LLaDA-Rec consistently outperforms both ID-based and state-of-the-art generative recommenders.
  arXiv  Detail & Related papers  (2025-11-09T07:12:15Z)
• Universal Item Tokenization for Transferable Generative Recommendation [89.42584009980676]
  We propose UTGRec, a universal item tokenization approach for transferable Generative Recommendation.<n>By devising tree-structured codebooks, we discretize content representations into corresponding codes for item tokenization.<n>For raw content reconstruction, we employ dual lightweight decoders to reconstruct item text and images from discrete representations.<n>For collaborative knowledge integration, we assume that co-occurring items are similar and integrate collaborative signals through co-occurrence alignment and reconstruction.
  arXiv  Detail & Related papers  (2025-04-06T08:07:49Z)
• An Autoregressive Text-to-Graph Framework for Joint Entity and Relation Extraction [4.194768796374315]
  We propose a novel method for joint entity and relation extraction from unstructured text by framing it as a conditional sequence generation problem. It generates a linearized graph where nodes represent text spans and edges represent relation triplets. Our method employs a transformer encoder-decoder architecture with pointing mechanism on a dynamic vocabulary of spans and relation types.
  arXiv  Detail & Related papers  (2024-01-02T18:32:14Z)
• Pushdown Layers: Encoding Recursive Structure in Transformer Language Models [86.75729087623259]
  Recursion is a prominent feature of human language, and fundamentally challenging for self-attention. This work introduces Pushdown Layers, a new self-attention layer. Transformers equipped with Pushdown Layers achieve dramatically better and 3-5x more sample-efficient syntactic generalization.
  arXiv  Detail & Related papers  (2023-10-29T17:27:18Z)
• Augmenting Sequential Recommendation with Pseudo-Prior Items via Reversely Pre-training Transformer [61.818320703583126]
  Sequential Recommendation characterizes the evolving patterns by modeling item sequences chronologically. Recent developments of transformer inspire the community to design effective sequence encoders. We introduce a new framework for textbfAugmenting textbfSequential textbfRecommendation with textbfPseudo-prior items(ASReP)
  arXiv  Detail & Related papers  (2021-05-02T18:06:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.