From Token to Item: Enhancing Large Language Models for Recommendation via Item-aware Attention Mechanism

• URL: http://arxiv.org/abs/2603.19693v1
• Date: Fri, 20 Mar 2026 06:56:59 GMT
• Title: From Token to Item: Enhancing Large Language Models for Recommendation via Item-aware Attention Mechanism
• Abstract summary: Large Language Models (LLMs) have recently gained increasing attention in the field of recommendation.<n>We revisit the role of tokens in recommendation and categorize their relations into two types: intra-item token relations and inter-item token relations.<n>We propose a novel framework with an item-aware attention mechanism (IAM) to enhance LLMs for recommendation.
• Score: 17.92505712532828
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Large Language Models (LLMs) have recently gained increasing attention in the field of recommendation. Existing LLM-based methods typically represent items as token sequences, and apply attention layers on these tokens to generate recommendations. However, by inheriting the standard attention mechanism, these methods focus on modeling token-level relations. This token-centric focus overlooks the item as the fundamental unit of recommendation, preventing existing methods from effectively capturing collaborative relations at the item level. In this work, we revisit the role of tokens in LLM-driven recommendation and categorize their relations into two types: (1) intra-item token relations, which present the content semantics of an item, e.g., name, color, and size; and (2) inter-item token relations, which encode collaborative relations across items. Building on these insights, we propose a novel framework with an item-aware attention mechanism (IAM) to enhance LLMs for recommendation. Specifically, IAM devises two complementary attention layers: (1) an intra-item attention layer, which restricts attention to tokens within the same item, modeling item content semantics; and (2) an inter-item attention layer, which attends exclusively to token relations across items, capturing item collaborative relations. Through this stacked design, IAM explicitly emphasizes items as the fundamental units in recommendation, enabling LLMs to effectively exploit item-level collaborative relations. Extensive experiments on several public datasets demonstrate the effectiveness of IAM in enhancing LLMs for personalized recommendation.

Related papers

• Making Collaborative Signals Count: Graph-Aware Large Language Models for Sequential Recommendation [14.98615627827995]
  We propose GALLM, a graph-aware LLM framework for sequential recommendation.<n>GALLM constructs a collaborative graph over text tokens and item tokens, and models three types of relations.<n> Experiments on four real-world benchmarks show that GALLM achieves the best performance among the compared baselines.
  arXiv  Detail & Related papers  (2026-08-12T15:38:05Z)
• HOMIE: Human-object Centric Video Personalization via Multimodal Intelligent Enhancement [49.760248537839715]
  Human-object centric video personalization (HOCVP) is a core task within subject-driven video generation.<n>We propose HOMIE, an HOCVP framework that tackles both inter- and intra-subject input settings in a unified manner.<n>We introduce global multimodal guidance within self-attention to better align MLLM-derived semantic features with VAE tokens.
  arXiv  Detail & Related papers  (2026-07-20T17:51:35Z)
• Fusion and Alignment Enhancement with Large Language Models for Tail-item Sequential Recommendation [17.14852109587492]
  Sequential Recommendation (SR) learns user preferences from their historical interaction sequences and provides personalized suggestions.<n>Large language models (LLMs) offer a promising solution by capturing semantic relationships between items.<n>We propose a Fusion and Alignment Enhancement framework with LLMs for Tail-item Sequential Recommendation (FAERec)
  arXiv  Detail & Related papers  (2026-04-04T11:19:08Z)
• Enhancing LLM-based Recommendation with Preference Hint Discovery from Knowledge Graph [26.065016082075264]
  We propose a preference hint discovery model based on the interaction-integrated knowledge graph.<n>We develop an instance-wise dual-attention mechanism to quantify the preference credibility of candidate attributes.
  arXiv  Detail & Related papers  (2026-01-26T03:20:42Z)
• HatLLM: Hierarchical Attention Masking for Enhanced Collaborative Modeling in LLM-based Recommendation [17.271853114690902]
  HatLLM is a hierarchical attention masking strategy for large language models (LLMs) for sequential recommendation.<n>HatLLM achieves significant performance gains (9.13% on average) over existing LLM-based methods.
  arXiv  Detail & Related papers  (2025-10-13T03:05:03Z)
• LLM2Rec: Large Language Models Are Powerful Embedding Models for Sequential Recommendation [49.78419076215196]
  Sequential recommendation aims to predict users' future interactions by modeling collaborative filtering (CF) signals from historical behaviors of similar users or items.<n>Traditional sequential recommenders rely on ID-based embeddings, which capture CF signals through high-order co-occurrence patterns.<n>Recent advances in large language models (LLMs) have motivated text-based recommendation approaches that derive item representations from textual descriptions.<n>We argue that an ideal embedding model should seamlessly integrate CF signals with rich semantic representations to improve both in-domain and out-of-domain recommendation performance.
  arXiv  Detail & Related papers  (2025-06-16T13:27:06Z)
• Knowledge Graph Completion with Relation-Aware Anchor Enhancement [50.50944396454757]
  We propose a relation-aware anchor enhanced knowledge graph completion method (RAA-KGC)<n>We first generate anchor entities within the relation-aware neighborhood of the head entity.<n>Then, by pulling the query embedding towards the neighborhoods of the anchors, it is tuned to be more discriminative for target entity matching.
  arXiv  Detail & Related papers  (2025-04-08T15:22:08Z)
• Universal Item Tokenization for Transferable Generative Recommendation [89.42584009980676]
  We propose UTGRec, a universal item tokenization approach for transferable Generative Recommendation.<n>By devising tree-structured codebooks, we discretize content representations into corresponding codes for item tokenization.<n>For raw content reconstruction, we employ dual lightweight decoders to reconstruct item text and images from discrete representations.<n>For collaborative knowledge integration, we assume that co-occurring items are similar and integrate collaborative signals through co-occurrence alignment and reconstruction.
  arXiv  Detail & Related papers  (2025-04-06T08:07:49Z)
• Enhancing Item Tokenization for Generative Recommendation through Self-Improvement [67.94240423434944]
  Generative recommendation systems are driven by large language models (LLMs)<n>Current item tokenization methods include using text descriptions, numerical strings, or sequences of discrete tokens.<n>We propose a self-improving item tokenization method that allows the LLM to refine its own item tokenizations during training process.
  arXiv  Detail & Related papers  (2024-12-22T21:56:15Z)
• LLM with Relation Classifier for Document-Level Relation Extraction [25.587850398830252]
  Large language models (LLMs) have created a new paradigm for natural language processing.<n>This paper investigates the causes of this performance gap, identifying the dispersion of attention by LLMs due to entity pairs without relations as a key factor.
  arXiv  Detail & Related papers  (2024-08-25T16:43:19Z)
• Recommender AI Agent: Integrating Large Language Models for Interactive Recommendations [53.76682562935373]
  We introduce an efficient framework called textbfInteRecAgent, which employs LLMs as the brain and recommender models as tools. InteRecAgent achieves satisfying performance as a conversational recommender system, outperforming general-purpose LLMs.
  arXiv  Detail & Related papers  (2023-08-31T07:36:44Z)
• Hierarchical Latent Relation Modeling for Collaborative Metric Learning [8.59872983871365]
  Collaborative Metric Learning (CML) recently emerged as a powerful paradigm for recommendation based on implicit feedback collaborative filtering. We present a hierarchical CML model that jointly captures latent user-item and item-item relations from implicit data. We empirically show the relevance of this joint relational modeling, by outperforming existing CML models on recommendation tasks on several real-world datasets.
  arXiv  Detail & Related papers  (2021-07-26T17:45:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.