Entriever: Energy-based Retriever for Knowledge-Grounded Dialog Systems

• URL: http://arxiv.org/abs/2506.00585v1
• Date: Sat, 31 May 2025 14:42:34 GMT
• Title: Entriever: Energy-based Retriever for Knowledge-Grounded Dialog Systems
• Authors: Yucheng Cai, Ke Li, Yi Huang, Junlan Feng, Zhijian Ou,
• Abstract summary: A retriever retrieves relevant knowledge pieces from a knowledge base given a context.<n>We propose Entriever, an energy-based retriever.<n>We show that Entriever substantially outperforms the strong cross-encoder baseline in knowledge retrieval tasks.
• Score: 20.612395472432798
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A retriever, which retrieves relevant knowledge pieces from a knowledge base given a context, is an important component in many natural language processing (NLP) tasks. Retrievers have been introduced in knowledge-grounded dialog systems to improve knowledge acquisition. In knowledge-grounded dialog systems, when conditioning on a given context, there may be multiple relevant and correlated knowledge pieces. However, knowledge pieces are usually assumed to be conditionally independent in current retriever models. To address this issue, we propose Entriever, an energy-based retriever. Entriever directly models the candidate retrieval results as a whole instead of modeling the knowledge pieces separately, with the relevance score defined by an energy function. We explore various architectures of energy functions and different training methods for Entriever, and show that Entriever substantially outperforms the strong cross-encoder baseline in knowledge retrieval tasks. Furthermore, we show that in semi-supervised training of knowledge-grounded dialog systems, Entriever enables effective scoring of retrieved knowledge pieces and significantly improves end-to-end performance of dialog systems.

Related papers

• Large Language Models are Limited in Out-of-Context Knowledge Reasoning [65.72847298578071]
  Large Language Models (LLMs) possess extensive knowledge and strong capabilities in performing in-context reasoning. This paper focuses on a significant aspect of out-of-context reasoning: Out-of-Context Knowledge Reasoning (OCKR), which is to combine multiple knowledge to infer new knowledge.
  arXiv  Detail & Related papers  (2024-06-11T15:58:59Z)
• Enhancing Knowledge Retrieval with Topic Modeling for Knowledge-Grounded Dialogue [0.6650227510403052]
  We propose an approach that utilizes topic modeling on the knowledge base to further improve retrieval accuracy. We also experiment with a large language model, ChatGPT, to take advantage of the improved retrieval performance.
  arXiv  Detail & Related papers  (2024-05-07T23:32:32Z)
• A Knowledge Plug-and-Play Test Bed for Open-domain Dialogue Generation [51.31429493814664]
  We present a benchmark named multi-source Wizard of Wikipedia for evaluating multi-source dialogue knowledge selection and response generation. We propose a new challenge, dialogue knowledge plug-and-play, which aims to test an already trained dialogue model on using new support knowledge from previously unseen sources.
  arXiv  Detail & Related papers  (2024-03-06T06:54:02Z)
• UniRQR: A Unified Model for Retrieval Decision, Query, and Response Generation in Internet-Based Knowledge Dialogue Systems [8.724141214921314]
  Knowledge-based dialogue systems with internet retrieval can be typically segmented into three tasks: Retrieval Decision, Query Generation, and Response Generation. Our work addresses this oversight by employing a single unified model facilitated by prompt and multi-task learning approaches. By integrating these functions, our system leverages the full potential of pre-trained models and reduces the complexity and costs associated with deploying multiple models.
  arXiv  Detail & Related papers  (2024-01-11T06:09:15Z)
• Multi-Grained Knowledge Retrieval for End-to-End Task-Oriented Dialog [42.088274728084265]
  Retrieving proper domain knowledge from an external database lies at the heart of end-to-end task-oriented dialog systems. Most existing systems blend knowledge retrieval with response generation and optimize them with direct supervision from reference responses. We propose to decouple knowledge retrieval from response generation and introduce a multi-grained knowledge retriever.
  arXiv  Detail & Related papers  (2023-05-17T12:12:46Z)
• UNTER: A Unified Knowledge Interface for Enhancing Pre-trained Language Models [100.4659557650775]
  We propose a UNified knowledge inTERface, UNTER, to provide a unified perspective to exploit both structured knowledge and unstructured knowledge. With both forms of knowledge injected, UNTER gains continuous improvements on a series of knowledge-driven NLP tasks.
  arXiv  Detail & Related papers  (2023-05-02T17:33:28Z)
• Knowledge-Grounded Dialogue Generation with a Unified Knowledge Representation [78.85622982191522]
  Existing systems perform poorly on unseen topics due to limited topics covered in the training data. We present PLUG, a language model that homogenizes different knowledge sources to a unified knowledge representation. It can achieve comparable performance with state-of-the-art methods under a fully-supervised setting.
  arXiv  Detail & Related papers  (2021-12-15T07:11:02Z)
• Retrieval-Free Knowledge-Grounded Dialogue Response Generation with Adapters [52.725200145600624]
  We propose KnowExpert to bypass the retrieval process by injecting prior knowledge into the pre-trained language models with lightweight adapters. Experimental results show that KnowExpert performs comparably with the retrieval-based baselines.
  arXiv  Detail & Related papers  (2021-05-13T12:33:23Z)
• Contextualized Knowledge-aware Attentive Neural Network: Enhancing Answer Selection with Knowledge [77.77684299758494]
  We extensively investigate approaches to enhancing the answer selection model with external knowledge from knowledge graph (KG) First, we present a context-knowledge interaction learning framework, Knowledge-aware Neural Network (KNN), which learns the QA sentence representations by considering a tight interaction with the external knowledge from KG and the textual information. To handle the diversity and complexity of KG information, we propose a Contextualized Knowledge-aware Attentive Neural Network (CKANN), which improves the knowledge representation learning with structure information via a customized Graph Convolutional Network (GCN) and comprehensively learns context-based and knowledge-based sentence representation via
  arXiv  Detail & Related papers  (2021-04-12T05:52:20Z)
• Unstructured Knowledge Access in Task-oriented Dialog Modeling using Language Inference, Knowledge Retrieval and Knowledge-Integrative Response Generation [44.184890645068485]
  Dialog systems enriched with external knowledge can handle user queries that are outside the scope of the supporting databases/APIs. We propose three subsystems, KDEAK, KnowleDgEFactor, and Ens-GPT, which form the pipeline for a task-oriented dialog system. Experimental results demonstrate that the proposed pipeline system outperforms the baseline and generates high-quality responses.
  arXiv  Detail & Related papers  (2021-01-15T11:24:32Z)
• Improving Commonsense Question Answering by Graph-based Iterative Retrieval over Multiple Knowledge Sources [26.256653692882715]
  How to engage commonsense effectively in question answering systems is still under exploration. We propose a novel question-answering method by integrating ConceptNet, Wikipedia, and the Cambridge Dictionary. We use a pre-trained language model to encode the question, retrieved knowledge and choices, and propose an answer choice-aware attention mechanism.
  arXiv  Detail & Related papers  (2020-11-05T08:50:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.