Matryoshka Hash Representations for Model-Aware Compact Semantic Retrieval

• URL: http://arxiv.org/abs/2609.07276v1
• Date: Mon, 07 Sep 2026 09:38:28 GMT
• Title: Matryoshka Hash Representations for Model-Aware Compact Semantic Retrieval
• Abstract summary: We introduce Matryoshka Hash Representations (MHR), a two-stage procedure that separates full-width training from prefix organization.<n>MHR reaches.5561 NDCG@10 and.6535 Recall@100 at 32 bytes, surpassing the best baseline of the same budget.<n>The code also strengthens two common pipelines: shortlisting candidates for full-precision reranking, and pruning a low-storage graph index.
• Score: 0.4859781642217615
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Retrieval-augmented generation (RAG) depends on dense retrieval: each document is stored as a learned vector, and a query is answered by finding its nearest neighbors in that vector space. Keeping one full-precision vector per document is the dominant index cost at corpus scale, so retrieval systems replace each vector with a short code of a few bytes---a step called quantization. Standard quantizers such as product quantization (PQ) pick the code that reconstructs the original vector most closely. A single code is even more useful if it serves several byte budgets at once: when its short prefixes are each directly searchable, a deployment can set its efficiency--quality operating point without re-encoding the corpus. But training all prefixes under one objective makes the early bits a compromise across budgets---short codes improve while the full-width code degrades. Quantization to low-bit representation, such as binary codes, further sharpens the conflict. We introduce Matryoshka Hash Representations (MHR), a two-stage procedure that separates full-width training from prefix organization. MHR first learns a longer binary code, then freezes the model and trains additional zero-initialized residual code adaptors for directly searchable prefixes. Documents are stored at one bit per coordinate, while queries keep continuous logits like PQ to attain sufficient expressivity. We implement the search process with FAISS FastScan. Trained on MS MARCO and zero-shot transferred to seven BEIR datasets, MHR reaches .5561 NDCG@10 and .6535 Recall@100 at 32 bytes, surpassing the best baseline of the same budget. The advantage is more pronounced in lower budgets. The same code also strengthens two common pipelines: shortlisting candidates for full-precision reranking, and pruning a low-storage graph index such as LEANN.

Related papers

• Fractal KV-Cache Archives: Lossless Symbolic Storage with In-Place Retrieval for Long-Context LLM Inference [0.0]
  Key-value ( KV) cache dominates the memory cost of long-context autoregressive inference.<n>We study how to store the resulting symbol stream once a position's KV state has been quantized to codebook indices.<n>A family of contractive iterated-map codes that serialize a symbol sequence into a sequence of low-dimensional real vectors is revisited.
  arXiv  Detail & Related papers  (2026-07-08T08:37:49Z)
• No More K-means: Single-Stage Sparse Coding for Efficient Multi-Vector Retrieval [51.43543998583709]
  Single-stage Sparse Retrieval (SSR) is a paradigm shift that replaces expensive clustering with efficient sparse coding.<n>It reduces indexing time by 15x compared to ColBERTv2, halves retrieval latency, and simultaneously improves retrieval performance.
  arXiv  Detail & Related papers  (2026-05-28T15:53:34Z)
• Efficient and Scalable Provenance Tracking for LLM-Generated Code Snippets [3.5312864406384485]
  Large language models (LLMs) for code completion and generation may reproduce training examples verbatim and without authorship attribution.<n> Classical fingerprint-based plagiarism detectors based on fingerprinting, such as Winnowing, remain highly effective.<n>We introduce a 300M- parameter encoder tailored for code retrieval, together with a hybrid two-stage provenance-tracking pipeline.
  arXiv  Detail & Related papers  (2026-05-27T14:12:17Z)
• Hierarchical Embedding Fusion for Retrieval-Augmented Code Generation [6.4453302264198165]
  We present Hierarchical Embedding Fusion (HEF), a two-stage approach to repository representation for code completion.<n>HEF achieves exact-match accuracy comparable to snippet-based retrieval baselines.<n>Compared to graph-based and iterative retrieval systems, HEF reduces median end-to-end latency by 13 to 26 times.
  arXiv  Detail & Related papers  (2026-02-04T14:56:11Z)
• FaTRQ: Tiered Residual Quantization for LLM Vector Search in Far-Memory-Aware ANNS Systems [16.221654013698963]
  FaTRQ is a far-memory-aware refinement system that eliminates the need to fetch full vectors from storage.<n>A custom accelerator is deployed in a CXL Type-2 device to perform low-latency refinement locally.<n>Together, FaTRQ improves the storage efficiency by 2.4$times$ and improves the throughput by up to 9$ times$ than SOTA GPU ANNS system.
  arXiv  Detail & Related papers  (2026-01-15T01:59:29Z)
• Qinco2: Vector Compression and Search with Improved Implicit Neural Codebooks [23.598010051403225]
  We introduce QINCo2, which improves the state-of-the-art reconstruction MSE by 34% for 16-byte vector compression on BigANN, and search accuracy by 24% with 8-byte encodings on Deep1M.<n>We conduct experiments on four datasets to evaluate QINCo2 for vector compression and billion-scale nearest neighbor search.
  arXiv  Detail & Related papers  (2025-01-06T15:20:22Z)
• SECRET: Towards Scalable and Efficient Code Retrieval via Segmented Deep Hashing [83.35231185111464]
  Deep learning has shifted the retrieval paradigm from lexical-based matching to encode source code and queries into vector representations.<n>Previous research proposes deep hashing-based methods, which generate hash codes for queries and code snippets and use Hamming distance for rapid recall of code candidates.<n>We propose a novel approach, which converts long hash codes calculated by existing deep hashing approaches into several short hash code segments through an iterative training strategy.
  arXiv  Detail & Related papers  (2024-12-16T12:51:35Z)
• Turning Trash into Treasure: Accelerating Inference of Large Language Models with Token Recycling [24.04649159686283]
  Speculative decoding is an approach to accelerate inference through a guess-and-verify paradigm.<n> Token Recycling stores candidate tokens in an adjacency matrix and employs a breadth-first-search algorithm.<n>It significantly outperforms existing train-free methods by 30% and even a widely recognized training method by 25%.
  arXiv  Detail & Related papers  (2024-08-16T12:20:56Z)
• Let the Code LLM Edit Itself When You Edit the Code [50.46536185784169]
  underlinetextbfPositional textbfIntegrity textbfEncoding (PIE)<n>PIE reduces computational overhead by over 85% compared to the standard full recomputation approach.<n>Results demonstrate that PIE reduces computational overhead by over 85% compared to the standard full recomputation approach.
  arXiv  Detail & Related papers  (2024-07-03T14:34:03Z)
• Linear-time Minimum Bayes Risk Decoding with Reference Aggregation [52.1701152610258]
  Minimum Bayes Risk (MBR) decoding is a text generation technique that has been shown to improve the quality of machine translations. It requires the pairwise calculation of a utility metric, which has quadratic complexity. We propose to approximate pairwise metric scores with scores calculated against aggregated reference representations.
  arXiv  Detail & Related papers  (2024-02-06T18:59:30Z)
• Tackling Long Code Search with Splitting, Encoding, and Aggregating [67.02322603435628]
  We propose a new baseline SEA (Split, Encode and Aggregate) for long code search. It splits long code into code blocks, encodes these blocks into embeddings, and aggregates them to obtain a comprehensive long code representation. With GraphCodeBERT as the encoder, SEA achieves an overall mean reciprocal ranking score of 0.785, which is 10.1% higher than GraphCodeBERT on the CodeSearchNet benchmark.
  arXiv  Detail & Related papers  (2022-08-24T02:27:30Z)
• Faster Person Re-Identification [68.22203008760269]
  We introduce a new solution for fast ReID by formulating a novel Coarse-to-Fine hashing code search strategy. It uses shorter codes to coarsely rank broad matching similarities and longer codes to refine only a few top candidates for more accurate instance ReID. Experimental results on 2 datasets show that our proposed method (CtF) is not only 8% more accurate but also 5x faster than contemporary hashing ReID methods.
  arXiv  Detail & Related papers  (2020-08-16T03:02:49Z)
• Pruning Neural Belief Propagation Decoders [77.237958592189]
  We introduce a method to tailor an overcomplete parity-check matrix to (neural) BP decoding using machine learning. We achieve performance within 0.27 dB and 1.5 dB of the ML performance while reducing the complexity of the decoder.
  arXiv  Detail & Related papers  (2020-01-21T12:05:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.