Decoupled Residual Quantization for Robust Semantic IDs in Recommendation

• URL: http://arxiv.org/abs/2606.01844v1
• Date: Mon, 01 Jun 2026 07:55:21 GMT
• Title: Decoupled Residual Quantization for Robust Semantic IDs in Recommendation
• Abstract summary: Poor quality tokenizer may come from codebook underutilization, unstable decision boundaries, or geometric distortion of the embedding space.<n>This paper develops a quantitative framework for diagnosing these failures through expected codeword overlap and effective codebook capacity.<n>As a proof of concept, we present Decoupled Residual Quantization (DRQ), which separates continuous geometry reconstruction from discrete distribution matching.
• Score: 6.85237130729423
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Semantic IDs represent items as shared discrete token sequences and have become a practical tool for recommendation and retrieval. Yet it remains difficult to tell why a tokenizer fails: poor quality may come from codebook underutilization, unstable decision boundaries, or geometric distortion of the embedding space. This paper develops a quantitative framework for diagnosing these failures through expected codeword overlap and effective codebook capacity. The former measures expected codeword confusion under retrieval-time perturbation, while the latter converts that confusion into an effective number of usable, well-separated codes. The framework links semantic boundary confusion to both code usage imbalance and Euclidean geometric constraints. As a proof of concept, we present Decoupled Residual Quantization (DRQ), which separates continuous geometry reconstruction from discrete distribution matching. Experiments on a large-scale industrial dataset show that Semantic ID quality is multi-objective: symbolic robustness, reconstruction fidelity, and behavior-aware soft matching each stress different aspects of a tokenizer. These downstream observations are based on one proprietary industrial dataset, so they should be read as a case study rather than a universal benchmark claim.

Related papers

• WIDE: Wildcard Inference with Dynamic Expansion for Cross-Modal Generative Retrieval [11.681276883454288]
  We propose Wildcard Inference with Dynamic Expansion (WIDE) to overcome the problem of forced hallucination in cross-modal retrieval.<n>During the decoding generation phase, Asymmetry-aware Wildcard Decoding (AWD) detects semantic blind spots and emits wildcards instead of forced deterministic identifiers.<n>Blind-Spot Re-ranking (BSR) evaluates the expanded candidate pool using a hybrid scoring mechanism that combines discrete generation confidence with continuous semantic similarity.
  arXiv  Detail & Related papers  (2026-09-03T08:55:05Z)
• Off-Policy Evaluation for Semantic ID Recommenders: Does the Model's Own Code Hierarchy Help? [1.438318734827691]
  Gene recommenders increasingly emit semantic IDs (SIDs)<n>Can the model's own SID tree serve as the action abstraction for that OPE?<n>A conditional bias bound links the coarsening bias to the knob quantizer's worst-case reconstruction residual.
  arXiv  Detail & Related papers  (2026-08-28T22:11:22Z)
• Closing the Indexing-Decoding Gap in Multimodal Generative Retrieval via Prefix Retention Optimization [68.48718919047127]
  Multimodal generative retrieval formulates multimodal retrieval as discrete identifier generation, eliminating the need for explicit similarity search over external embeddings.<n>Existing approaches construct identifiers via residual quantization and decode them with trie-constrained beam search.<n>This combination introduces an indexing-decoding gap: identifier learning objectives, including reconstruction and contrastive losses, do not explicitly enforce prefix discriminability during decoding.<n>We propose PRO, a unified framework comprising three mechanisms: (i) prefix ranking distillation aligns quantized prefixes with those induced by pre-quantization embeddings using a listwise loss; (ii) vocabulary scheduling increases codebook
  arXiv  Detail & Related papers  (2026-06-08T09:15:47Z)
• Asymmetric Generative Recommendation via Multi-Expert Projection and Multi-Faceted Hierarchical Quantization [39.92412918837748]
  AsymRec is an asymmetric continuous-discrete framework that decouples input and output representations.<n>It consistently outperforms state-of-the-art generative recommenders by an average of 15.8 %.
  arXiv  Detail & Related papers  (2026-05-14T07:55:43Z)
• Efficient Test-Time Inference via Deterministic Exploration of Truncated Decoding Trees [68.04613115686509]
  Self-consistency boosts inference-time performance by sampling multiple reasoning traces in parallel and voting.<n>We propose Distinct Leafion (DLE), a deterministic decoding method that treats truncated sampling as a pruned tree and systematically enumerates distinct leaves instead of sampling with replacement.<n>DLE explores higher-quality reasoning traces, yielding better performance on math, coding, and general reasoning tasks.
  arXiv  Detail & Related papers  (2026-04-22T12:42:03Z)
• Bi-Lipschitz Autoencoder With Injectivity Guarantee [14.99578789813769]
  Bi-Lipschitz Autoencoder (BLAE) is an injective regularization scheme based on a separation criterion to eliminate pathological local minima.<n>BLAE consistently outperforms existing methods in preserving manifold structure while remaining resilient to sampling sparsity and distribution shifts.
  arXiv  Detail & Related papers  (2026-04-08T05:40:00Z)
• Overcoming Joint Intractability with Lossless Hierarchical Speculative Decoding [58.92526489742584]
  We propose provably lossless.<n> verification method that significantly boosts the expected number of accepted tokens.<n>We show that HSD yields consistent improvements in acceptance rates across diverse model families and benchmarks.
  arXiv  Detail & Related papers  (2026-01-09T11:10:29Z)
• Accelerate Speculative Decoding with Sparse Computation in Verification [49.74839681322316]
  Speculative decoding accelerates autoregressive language model inference by verifying multiple draft tokens in parallel.<n>Existing sparsification methods are designed primarily for standard token-by-token autoregressive decoding.<n>We propose a sparse verification framework that jointly sparsifies attention, FFN, and MoE components during the verification stage to reduce the dominant computation cost.
  arXiv  Detail & Related papers  (2025-12-26T07:53:41Z)
• Taming Imperfect Process Verifiers: A Sampling Perspective on Backtracking [54.43083499412643]
  Test-time algorithms that combine the generative power of language models with process verifiers offer a promising lever for eliciting new reasoning capabilities.<n>We introduce a new process-guided test-time sampling algorithm, VGB, which uses theoretically grounded backtracking to achieve provably better robustness to verifier errors.
  arXiv  Detail & Related papers  (2025-10-03T16:21:14Z)
• Exploiting Discriminative Codebook Prior for Autoregressive Image Generation [54.14166700058777]
  token-based autoregressive image generation systems first tokenize images into sequences of token indices with a codebook, and then model these sequences in an autoregressive paradigm.<n>While autoregressive generative models are trained only on index values, the prior encoded in the codebook, which contains rich token similarity information, is not exploited.<n>Recent studies have attempted to incorporate this prior by performing naive k-means clustering on the tokens, helping to facilitate the training of generative models with a reduced codebook.<n>We propose the Discriminative Codebook Prior Extractor (DCPE) as an alternative to k-means
  arXiv  Detail & Related papers  (2025-08-14T15:00:00Z)
• SaraCoder: Orchestrating Semantic and Structural Cues for Resource-Optimized Repository-Level Code Completion [34.41683042851225]
  We propose a resource-optimized retrieval augmentation method, SaraCoder.<n>It maximizes information diversity and representativeness in a limited context window.<n>Our work proves that systematically refining retrieval results across multiple dimensions provides a new paradigm for building more accurate and resource-optimized repository-level code completion systems.
  arXiv  Detail & Related papers  (2025-08-13T11:56:05Z)
• SparseCL: Sparse Contrastive Learning for Contradiction Retrieval [87.02936971689817]
  Contradiction retrieval refers to identifying and extracting documents that explicitly disagree with or refute the content of a query. Existing methods such as similarity search and crossencoder models exhibit significant limitations. We introduce SparseCL that leverages specially trained sentence embeddings designed to preserve subtle, contradictory nuances between sentences.
  arXiv  Detail & Related papers  (2024-06-15T21:57:03Z)
• Advanced Detection of Source Code Clones via an Ensemble of Unsupervised Similarity Measures [0.0]
  This research introduces a novel ensemble learning approach for code similarity assessment. The key idea is that the strengths of a diverse set of similarity measures can complement each other and mitigate individual weaknesses.
  arXiv  Detail & Related papers  (2024-05-03T13:42:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.