Exploring Bottom-Up Clustering for Creating Semantic IDs

• URL: http://arxiv.org/abs/2609.08310v1
• Date: Tue, 08 Sep 2026 06:32:57 GMT
• Title: Exploring Bottom-Up Clustering for Creating Semantic IDs
• Abstract summary: We present an algorithm for generating Semantic IDs that ensure the identifiers are both unique and preserve the structure of the original embedding.<n>Key to our work is the use of bottom-up clustering to preserve local structure in the embedding space, improving the clustering quality of the resulting Semantic IDs.
• Score: 6.910490819628779
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The success of generative retrieval has largely been attributed to the use of Semantic IDs, which improve over arbitrary item-level identifiers such as hashes by capturing the semantics of items. The main challenges faced when constructing Semantic IDs, however, is in mapping each identifier to a unique product and capturing information valuable to downstream tasks. Past works have appended additional codewords to de-duplicate item identifiers and utilized residual quantization to create hierarchical clusters. In this work, we present an algorithm for generating Semantic IDs that ensure the identifiers are both unique and preserve the structure of the original embedding. Key to our work is the use of bottom-up clustering to preserve local structure in the embedding space, improving the clustering quality of the resulting Semantic IDs and their utility for downstream generative retrieval.

Related papers

• C2T-ID: Converting Semantic Codebooks to Textual Document Identifiers for Generative Search [73.61009656398384]
  We propose C2T-ID, which builds semantic numerical docid via hierarchical clustering.<n>C2T-ID significantly outperforms atomic, semantic codebook, and pure-text docid baselines.
  arXiv  Detail & Related papers  (2025-10-22T04:05:38Z)
• MLLM-Driven Semantic Identifier Generation for Generative Cross-Modal Retrieval [7.524529523498721]
  We propose a vocabulary-efficient identifier generation framework that prompts MLLMs to generate Structured Semantic Identifiers from image-caption pairs.<n>These identifiers are composed of concept-level tokens such as objects and actions, naturally aligning with the model's generation space.<n>We also introduce a Rationale-Guided Supervision Strategy, prompting the model to produce a one-sentence explanation alongside each identifier.
  arXiv  Detail & Related papers  (2025-09-22T05:23:06Z)
• Purely Semantic Indexing for LLM-based Generative Recommendation and Retrieval [28.366331215978445]
  We propose purely semantic indexing to generate unique, semantic-preserving IDs without appending non-semantic tokens.<n>We enable unique ID assignment by relaxing the strict nearest-centroid selection and introduce two model-agnostic algorithms.
  arXiv  Detail & Related papers  (2025-09-19T21:59:55Z)
• Order-agnostic Identifier for Large Language Model-based Generative Recommendation [94.37662915542603]
  Items are assigned identifiers for Large Language Models (LLMs) to encode user history and generate the next item.<n>Existing approaches leverage either token-sequence identifiers, representing items as discrete token sequences, or single-token identifiers, using ID or semantic embeddings.<n>We propose SETRec, which leverages semantic tokenizers to obtain order-agnostic multi-dimensional tokens.
  arXiv  Detail & Related papers  (2025-02-15T15:25:38Z)
• Generative Retrieval as Multi-Vector Dense Retrieval [71.75503049199897]
  Generative retrieval generates identifiers of relevant documents in an end-to-end manner. Prior work has demonstrated that generative retrieval with atomic identifiers is equivalent to single-vector dense retrieval. We show that generative retrieval and multi-vector dense retrieval share the same framework for measuring the relevance to a query of a document.
  arXiv  Detail & Related papers  (2024-03-31T13:29:43Z)
• Summarization-Based Document IDs for Generative Retrieval with Language Models [65.11811787587403]
  We introduce summarization-based document IDs, in which each document's ID is composed of an extractive summary or abstractive keyphrases. We show that using ACID improves top-10 and top-20 recall by 15.6% and 14.4% (relative) respectively. We also observed that extractive IDs outperformed abstractive IDs on Wikipedia articles in NQ but not the snippets in MSMARCO.
  arXiv  Detail & Related papers  (2023-11-14T23:28:36Z)
• Language Models As Semantic Indexers [78.83425357657026]
  We introduce LMIndexer, a self-supervised framework to learn semantic IDs with a generative language model. We show the high quality of the learned IDs and demonstrate their effectiveness on three tasks including recommendation, product search, and document retrieval.
  arXiv  Detail & Related papers  (2023-10-11T18:56:15Z)
• Multiview Identifiers Enhanced Generative Retrieval [78.38443356800848]
  generative retrieval generates identifier strings of passages as the retrieval target. We propose a new type of identifier, synthetic identifiers, that are generated based on the content of a passage. Our proposed approach performs the best in generative retrieval, demonstrating its effectiveness and robustness.
  arXiv  Detail & Related papers  (2023-05-26T06:50:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.