Dynamic Injection of Entity Knowledge into Dense Retrievers

• URL: http://arxiv.org/abs/2507.03922v2
• Date: Mon, 08 Sep 2025 04:27:20 GMT
• Title: Dynamic Injection of Entity Knowledge into Dense Retrievers
• Authors: Ikuya Yamada, Ryokan Ri, Takeshi Kojima, Yusuke Iwasawa, Yutaka Matsuo,
• Abstract summary: Knowledgeable Passage Retriever (KPR) is a BERT-based retriever enhanced with a context-entity attention layer and dynamically updatable entity embeddings.<n> Experiments on three datasets demonstrate that KPR consistently improves retrieval accuracy.
• Score: 52.357666366609415
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Dense retrievers often struggle with queries involving less-frequent entities due to their limited entity knowledge. We propose the Knowledgeable Passage Retriever (KPR), a BERT-based retriever enhanced with a context-entity attention layer and dynamically updatable entity embeddings. This design enables KPR to incorporate external entity knowledge without retraining. Experiments on three datasets demonstrate that KPR consistently improves retrieval accuracy, with particularly large gains on the EntityQuestions dataset. When built on the off-the-shelf bge-base retriever, KPR achieves state-of-the-art performance among similarly sized models on two datasets. Models and code are released at https://github.com/knowledgeable-embedding/knowledgeable-embedding.

Related papers

• Logic-Oriented Retriever Enhancement via Contrastive Learning [15.39205142672531]
  LORE (Logic ORiented Retriever Enhancement) introduces fine-grained contrastive learning to activate latent logical analysis capacity.<n>LORE requires no external upervision, resources, or pre-retrieval analysis, remains index-compatible, and consistently improves retrieval utility and downstream generation.
  arXiv  Detail & Related papers  (2026-02-01T09:30:04Z)
• IMRNNs: An Efficient Method for Interpretable Dense Retrieval via Embedding Modulation [10.74445059832999]
  Interpretability in black-box dense retrievers remains a central challenge in Retrieval-Augmented Generation.<n>We propose Interpretable Modular Retrieval Neural Networks (IMRNNs), a lightweight framework that augments any dense retriever with dynamic, bidirectional modulation at inference time.
  arXiv  Detail & Related papers  (2026-01-27T22:02:51Z)
• Reasoning-enhanced Query Understanding through Decomposition and Interpretation [87.56450566014625]
  ReDI is a Reasoning-enhanced approach for query understanding through Decomposition and Interpretation.<n>We compiled a large-scale dataset of real-world complex queries from a major search engine.<n> Experiments on BRIGHT and BEIR demonstrate that ReDI consistently surpasses strong baselines in both sparse and dense retrieval paradigms.
  arXiv  Detail & Related papers  (2025-09-08T10:58:42Z)
• Revela: Dense Retriever Learning via Language Modeling [85.12131321155486]
  We introduce Revela, a unified training framework for self-supervised retriever learning via language modeling.<n>We evaluate Revela on both general-domain (BEIR) and domain-specific (CoIR) benchmarks across various retriever backbones.
  arXiv  Detail & Related papers  (2025-06-19T19:13:59Z)
• LogiCoL: Logically-Informed Contrastive Learning for Set-based Dense Retrieval [54.2765452810124]
  We introduce LogiCoL, a logically-informed contrastive learning objective for dense retrievers.<n>We show that models trained with LogiCoL yield improvement both in terms of retrieval performance and logical consistency in the results.
  arXiv  Detail & Related papers  (2025-05-26T07:00:32Z)
• Pre-training vs. Fine-tuning: A Reproducibility Study on Dense Retrieval Knowledge Acquisition [28.48078856765935]
  Dense retrievers utilize pre-trained backbone language models (e.g., BERT, LLaMA) that are fine-tuned via contrastive learning to perform the task of encoding text into sense representations.<n>Recent research has questioned the role of fine-tuning vs. that of pre-training within dense retrievers.<n>Our study confirms that in DPR tuning, pre-trained knowledge underpins retrieval performance, with fine-tuning primarily adjusting neuron activation rather than reorganizing knowledge.
  arXiv  Detail & Related papers  (2025-05-12T01:24:00Z)
• ReasonIR: Training Retrievers for Reasoning Tasks [139.54343970560103]
  ReasonIR-8B is the first retriever specifically trained for general reasoning tasks.<n>It achieves a new state-of-the-art of 29.9 nDCG@10 without reranker and 36.9 nDCG@10 with reranker on BRIGHT, a widely-used information retrieval benchmark.
  arXiv  Detail & Related papers  (2025-04-29T09:49:28Z)
• Pseudo Relevance Feedback is Enough to Close the Gap Between Small and Large Dense Retrieval Models [29.934928091542375]
  Scaling dense retrievers to larger large language model (LLM) backbones has been a dominant strategy for improving their retrieval effectiveness.<n>We introduce PromptPRF, a feature-based pseudo-relevance feedback (PRF) framework that enables small LLM-based dense retrievers to achieve effectiveness comparable to much larger models.
  arXiv  Detail & Related papers  (2025-03-19T04:30:20Z)
• Boot and Switch: Alternating Distillation for Zero-Shot Dense Retrieval [50.47192086219752]
  $texttABEL$ is a simple but effective unsupervised method to enhance passage retrieval in zero-shot settings. By either fine-tuning $texttABEL$ on labelled data or integrating it with existing supervised dense retrievers, we achieve state-of-the-art results.
  arXiv  Detail & Related papers  (2023-11-27T06:22:57Z)
• Revisiting Sparse Retrieval for Few-shot Entity Linking [33.15662306409253]
  We propose an ELECTRA-based keyword extractor to denoise the mention context and construct a better query expression. For training the extractor, we propose a distant supervision method to automatically generate training data based on overlapping tokens between mention contexts and entity descriptions. Experimental results on the ZESHEL dataset demonstrate that the proposed method outperforms state-of-the-art models by a significant margin across all test domains.
  arXiv  Detail & Related papers  (2023-10-19T03:51:10Z)
• Multi-Grained Knowledge Retrieval for End-to-End Task-Oriented Dialog [42.088274728084265]
  Retrieving proper domain knowledge from an external database lies at the heart of end-to-end task-oriented dialog systems. Most existing systems blend knowledge retrieval with response generation and optimize them with direct supervision from reference responses. We propose to decouple knowledge retrieval from response generation and introduce a multi-grained knowledge retriever.
  arXiv  Detail & Related papers  (2023-05-17T12:12:46Z)
• AugTriever: Unsupervised Dense Retrieval and Domain Adaptation by Scalable Data Augmentation [44.93777271276723]
  We propose two approaches that enable annotation-free and scalable training by creating pseudo querydocument pairs. The query extraction method involves selecting salient spans from the original document to generate pseudo queries. The transferred query generation method utilizes generation models trained for other NLP tasks, such as summarization, to produce pseudo queries.
  arXiv  Detail & Related papers  (2022-12-17T10:43:25Z)
• Learning to Retrieve Passages without Supervision [58.31911597824848]
  Dense retrievers for open-domain question answering (ODQA) have been shown to achieve impressive performance by training on large datasets of question-passage pairs. We investigate whether dense retrievers can be learned in a self-supervised fashion, and applied effectively without any annotations.
  arXiv  Detail & Related papers  (2021-12-14T19:18:08Z)
• You Only Need One Model for Open-domain Question Answering [26.582284346491686]
  Recent works for Open-domain Question Answering refer to an external knowledge base using a retriever model. We propose casting the retriever and the reranker as hard-attention mechanisms applied sequentially within the transformer architecture. We evaluate our model on Natural Questions and TriviaQA open datasets and our model outperforms the previous state-of-the-art model by 1.0 and 0.7 exact match scores.
  arXiv  Detail & Related papers  (2021-12-14T13:21:11Z)
• AutoRC: Improving BERT Based Relation Classification Models via Architecture Search [50.349407334562045]
  BERT based relation classification (RC) models have achieved significant improvements over the traditional deep learning models. No consensus can be reached on what is the optimal architecture. We design a comprehensive search space for BERT based RC models and employ neural architecture search (NAS) method to automatically discover the design choices.
  arXiv  Detail & Related papers  (2020-09-22T16:55:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.