Unified Semantic and ID Representation Learning for Deep Recommenders

• URL: http://arxiv.org/abs/2502.16474v1
• Date: Sun, 23 Feb 2025 07:17:28 GMT
• Title: Unified Semantic and ID Representation Learning for Deep Recommenders
• Authors: Guanyu Lin, Zhigang Hua, Tao Feng, Shuang Yang, Bo Long, Jiaxuan You,
• Abstract summary: We propose a Unified Semantic and ID Representation Learning framework.<n>In our framework, ID tokens capture unique item attributes, while semantic tokens represent shared, transferable characteristics.<n>Our framework integrates cosine similarity in earlier layers and Euclidean distance in the final layer to optimize representation learning.
• Score: 28.709935854073535
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Effective recommendation is crucial for large-scale online platforms. Traditional recommendation systems primarily rely on ID tokens to uniquely identify items, which can effectively capture specific item relationships but suffer from issues such as redundancy and poor performance in cold-start scenarios. Recent approaches have explored using semantic tokens as an alternative, yet they face challenges, including item duplication and inconsistent performance gains, leaving the potential advantages of semantic tokens inadequately examined. To address these limitations, we propose a Unified Semantic and ID Representation Learning framework that leverages the complementary strengths of both token types. In our framework, ID tokens capture unique item attributes, while semantic tokens represent shared, transferable characteristics. Additionally, we analyze the role of cosine similarity and Euclidean distance in embedding search, revealing that cosine similarity is more effective in decoupling accumulated embeddings, while Euclidean distance excels in distinguishing unique items. Our framework integrates cosine similarity in earlier layers and Euclidean distance in the final layer to optimize representation learning. Experiments on three benchmark datasets show that our method significantly outperforms state-of-the-art baselines, with improvements ranging from 6\% to 17\% and a reduction in token size by over 80%. These results demonstrate the effectiveness of combining ID and semantic tokenization to enhance the generalization ability of recommender systems.

Related papers

• "Principal Components" Enable A New Language of Images [79.45806370905775]
  We introduce a novel visual tokenization framework that embeds a provable PCA-like structure into the latent token space. Our approach achieves state-of-the-art reconstruction performance and enables better interpretability to align with the human vision system.
  arXiv  Detail & Related papers  (2025-03-11T17:59:41Z)
• Order-agnostic Identifier for Large Language Model-based Generative Recommendation [94.37662915542603]
  Items are assigned identifiers for Large Language Models (LLMs) to encode user history and generate the next item.<n>Existing approaches leverage either token-sequence identifiers, representing items as discrete token sequences, or single-token identifiers, using ID or semantic embeddings.<n>We propose SETRec, which leverages semantic tokenizers to obtain order-agnostic multi-dimensional tokens.
  arXiv  Detail & Related papers  (2025-02-15T15:25:38Z)
• Attention with Dependency Parsing Augmentation for Fine-Grained Attribution [26.603281615221505]
  We develop a fine-grained attribution mechanism that provides supporting evidence from retrieved documents for every answer span.<n>Existing attribution methods rely on model-internal similarity metrics between responses and documents, such as saliency scores and hidden state similarity.<n>We propose two techniques applicable to all model-internals-based methods. First, we aggregate token-wise evidence through set union operations, preserving the granularity of representations.<n>Second, we enhance the attributor by integrating dependency parsing to enrich the semantic completeness of target spans.
  arXiv  Detail & Related papers  (2024-12-16T03:12:13Z)
• Dynamic Token Selection for Aerial-Ground Person Re-Identification [0.36832029288386137]
  We propose a novel Dynamic Token Selective Transformer (DTST) tailored for AGPReID.<n>We segment the input image into multiple tokens, with each token representing a unique region or feature within the image.<n>Using a Top-k strategy, we extract the k most significant tokens that contain vital information essential for identity recognition.
  arXiv  Detail & Related papers  (2024-11-30T11:07:11Z)
• STORE: Streamlining Semantic Tokenization and Generative Recommendation with A Single LLM [59.08493154172207]
  We propose a unified framework to streamline the semantic tokenization and generative recommendation process. We formulate semantic tokenization as a text-to-token task and generative recommendation as a token-to-token task, supplemented by a token-to-text reconstruction task and a text-to-token auxiliary task. All these tasks are framed in a generative manner and trained using a single large language model (LLM) backbone.
  arXiv  Detail & Related papers  (2024-09-11T13:49:48Z)
• EAGER: Two-Stream Generative Recommender with Behavior-Semantic Collaboration [63.112790050749695]
  We introduce EAGER, a novel generative recommendation framework that seamlessly integrates both behavioral and semantic information. We validate the effectiveness of EAGER on four public benchmarks, demonstrating its superior performance compared to existing methods.
  arXiv  Detail & Related papers  (2024-06-20T06:21:56Z)
• Semantic Equitable Clustering: A Simple and Effective Strategy for Clustering Vision Tokens [57.37893387775829]
  We introduce a fast and balanced clustering method, named textbfSemantic textbfEquitable textbfClustering (SEC) SEC clusters tokens based on their global semantic relevance in an efficient, straightforward manner. We propose a versatile vision backbone, SECViT, to serve as a vision language connector.
  arXiv  Detail & Related papers  (2024-05-22T04:49:00Z)
• Subobject-level Image Tokenization [60.80949852899857]
  Patch-based image tokenization ignores the morphology of the visual world. Inspired by subword tokenization, we introduce subobject-level adaptive token segmentation. We show that subobject tokenization enables faster convergence and better generalization while using fewer visual tokens.
  arXiv  Detail & Related papers  (2024-02-22T06:47:44Z)
• MST: Adaptive Multi-Scale Tokens Guided Interactive Segmentation [8.46894039954642]
  We propose a novel multi-scale token adaptation algorithm for interactive segmentation. By performing top-k operations across multi-scale tokens, the computational complexity is greatly simplified. We also propose a token learning algorithm based on contrastive loss.
  arXiv  Detail & Related papers  (2024-01-09T07:59:42Z)
• Attribute-Aware Deep Hashing with Self-Consistency for Large-Scale Fine-Grained Image Retrieval [65.43522019468976]
  We propose attribute-aware hashing networks with self-consistency for generating attribute-aware hash codes. We develop an encoder-decoder structure network of a reconstruction task to unsupervisedly distill high-level attribute-specific vectors. Our models are equipped with a feature decorrelation constraint upon these attribute vectors to strengthen their representative abilities.
  arXiv  Detail & Related papers  (2023-11-21T08:20:38Z)
• Learning Context-aware Classifier for Semantic Segmentation [88.88198210948426]
  In this paper, contextual hints are exploited via learning a context-aware classifier. Our method is model-agnostic and can be easily applied to generic segmentation models. With only negligible additional parameters and +2% inference time, decent performance gain has been achieved on both small and large models.
  arXiv  Detail & Related papers  (2023-03-21T07:00:35Z)
• Self-supervised asymmetric deep hashing with margin-scalable constraint for image retrieval [3.611160663701664]
  We propose a novel self-supervised asymmetric deep hashing method with a margin-scalable constraint(SADH) approach for image retrieval. SADH implements a self-supervised network to preserve semantic information in a semantic feature map and a semantic code map for the semantics of the given dataset. For the feature learning part, a new margin-scalable constraint is employed for both highly-accurate construction of pairwise correlations in the hamming space and a more discriminative hash code representation.
  arXiv  Detail & Related papers  (2020-12-07T16:09:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.