H+ Embedding: Harmonizing Global and Token-Level Retrieval with Context-Dependent Phrases

• URL: http://arxiv.org/abs/2608.00065v2
• Date: Thu, 06 Aug 2026 07:56:44 GMT
• Title: H+ Embedding: Harmonizing Global and Token-Level Retrieval with Context-Dependent Phrases
• Abstract summary: H+ Embedding predicts variable-length phrase partitions, preserves uncovered tokens as singletons, and applies importance-guided unit selection with weighted MaxSim interaction.<n>Across 16 scientific, medical, and bilingual tasks, its phrase retrieval branch exceeds the global retrieval branch by 6.91 macro nDCG@10.<n>It also nearly matches Token while using 13.7% fewer document vectors and outperforms content-independent grouping rules under moderate vector budgets.
• Score: 20.2337258233724
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Terminology-intensive retrieval, especially in medical settings, depends on preserving multi-word entities, abbreviations, numerical constraints, and compositional concepts. However, existing representations lie at two extremes: single-vector retrievers often over-compress local relevance signals, while token-level late interaction retains every tokenizer subword at substantial indexing, storage, and scoring cost. This mismatch raises a natural question: can context-dependent phrases provide a useful retrieval unit between global vectors and tokens? We introduce H+ Embedding, a unified multi-granularity retriever that predicts variable-length phrase partitions, preserves uncovered tokens as singletons, and applies importance-guided unit selection with weighted MaxSim interaction. Across 16 scientific, medical, and bilingual tasks, its phrase retrieval branch exceeds the global retrieval branch by 6.91 macro nDCG@10. It also nearly matches Token while using 13.7% fewer document vectors and outperforms content-independent grouping rules under moderate vector budgets. Context-dependent phrase interaction therefore provides an intermediate quality-cost point between global compression and token-level interaction for practical retrieval systems.

Related papers

• Better, Stronger, Faster, and Broader: Structured All-Mask Prediction for MLLM-Based Segmentation [10.813064862132379]
  STAMP combines strong referring and reasoning segmentation with preserved multimodal ability and efficient inference.<n>It generates a target list with explicit IDs and optional boxes, binds these IDs to a shared multi-class mask space, and jointly predicts all targets in one non-autoregressive pass.<n>A single unified checkpoint retains STAMP's referring and reasoning capabilities while extending to open-vocabulary semantic, instance-aware, and remote-sensing small-target segmentation.
  arXiv  Detail & Related papers  (2026-08-03T18:42:11Z)
• Sparse Attention for Dense Open-Vocabulary Prediction in CLIP [54.45759517856271]
  We study an inference-time substitution of the row-wise softmax in the final visual self-attention layers.<n>Because entmax applies a data-dependent threshold that maps low scores exactly to zero, it acts as an implicit denoiser.<n>We evaluate on open-vocabulary tasks-dense semantic segmentation (Pascal VOC, Pascal Context, ADE20K) and fine-grained retrieval (FG-OVD)
  arXiv  Detail & Related papers  (2026-07-08T08:30:48Z)
• Improving Long-Context Retrieval with Multi-Prefix Embedding [64.08968715893504]
  Multi-Prefix Embedding partitions a document into chunks separated by EOS tokens, encodes the full sequence in a single causal forward pass, and extracts one embedding at each prefix boundary.<n>MPE retains cross-chunk context, enables chunk-level MaxSim matching, and trains with only document-level relevance labels.
  arXiv  Detail & Related papers  (2026-06-22T17:31:02Z)
• TIDE: Every Layer Knows the Token Beneath the Context [21.609102126208608]
  We propose TIDE, which augments the standard transformer with EmbeddingMemory.<n>We theoretically and empirically establish the benefits of TIDE in addressing the issues associated with single-token identity injection.
  arXiv  Detail & Related papers  (2026-05-07T13:16:18Z)
• Beyond Semantic Similarity: Rethinking Retrieval for Agentic Search via Direct Corpus Interaction [127.64173950476702]
  We study direct corpus interaction (DCI), where an agent searches the raw corpus directly with general-purpose terminal tools.<n>This approach requires no offline indexing and adapts naturally to evolving local corpora.<n>Across IR benchmarks and end-to-end agentic search tasks, this simple setup substantially outperforms strong sparse, dense, and reranking baselines.
  arXiv  Detail & Related papers  (2026-05-03T19:13:11Z)
• Multi-Vector Index Compression in Any Modality [73.7330345057813]
  Late interaction has emerged as a dominant paradigm for information retrieval in text, images, visual documents, and videos.<n>We introduce four approaches for index compression: sequence resizing, memory tokens, hierarchical pooling, and a novel attention-guided clustering (AGC)<n>AGC uses an attention-guided mechanism to identify the most semantically salient regions of a document as cluster centroids and to weight token aggregation.
  arXiv  Detail & Related papers  (2026-02-24T18:57:33Z)
• Beyond RAG for Agent Memory: Retrieval by Decoupling and Aggregation [22.803751188961865]
  We argue retrieval should move beyond similarity matching and instead operate over latent components.<n>We propose xMemory, which builds a hierarchy of intact units and maintains a searchable high-level node organisation.
  arXiv  Detail & Related papers  (2026-02-02T12:04:58Z)
• A General and Flexible Multi-concept Parsing Framework for Multilingual Semantic Matching [60.51839859852572]
  We propose to resolve the text into multi concepts for multilingual semantic matching to liberate the model from the reliance on NER models. We conduct comprehensive experiments on English datasets QQP and MRPC, and Chinese dataset Medical-SM.
  arXiv  Detail & Related papers  (2024-03-05T13:55:16Z)
• mCL-NER: Cross-Lingual Named Entity Recognition via Multi-view Contrastive Learning [54.523172171533645]
  Cross-lingual named entity recognition (CrossNER) faces challenges stemming from uneven performance due to the scarcity of multilingual corpora. We propose Multi-view Contrastive Learning for Cross-lingual Named Entity Recognition (mCL-NER) Our experiments on the XTREME benchmark, spanning 40 languages, demonstrate the superiority of mCL-NER over prior data-driven and model-based approaches.
  arXiv  Detail & Related papers  (2023-08-17T16:02:29Z)
• Multi-Vector Retrieval as Sparse Alignment [21.892007741798853]
  We propose a novel multi-vector retrieval model that learns sparsified pairwise alignments between query and document tokens. We learn the sparse unary saliences with entropy-regularized linear programming, which outperforms other methods to achieve sparsity. Our model often produces interpretable alignments and significantly improves its performance when from larger language models.
  arXiv  Detail & Related papers  (2022-11-02T16:49:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.