Improving Factual Consistency for Knowledge-Grounded Dialogue Systems
via Knowledge Enhancement and Alignment
- URL: http://arxiv.org/abs/2310.08372v3
- Date: Fri, 3 Nov 2023 07:26:42 GMT
- Title: Improving Factual Consistency for Knowledge-Grounded Dialogue Systems
via Knowledge Enhancement and Alignment
- Authors: Boyang Xue and Weichao Wang and Hongru Wang and Fei Mi and Rui Wang
and Yasheng Wang and Lifeng Shang and Xin Jiang and Qun Liu and Kam-Fai Wong
- Abstract summary: Pretrained language models (PLMs) based knowledge-grounded dialogue systems are prone to generate responses that are factually inconsistent with the provided knowledge source.
Inspired by previous work which identified that feed-forward networks (FFNs) within Transformers are responsible for factual knowledge expressions, we investigate two methods to efficiently improve the factual expression capability.
- Score: 77.56326872997407
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Pretrained language models (PLMs) based knowledge-grounded dialogue systems
are prone to generate responses that are factually inconsistent with the
provided knowledge source. In such inconsistent responses, the dialogue models
fail to accurately express the external knowledge they rely upon. Inspired by
previous work which identified that feed-forward networks (FFNs) within
Transformers are responsible for factual knowledge expressions, we investigate
two methods to efficiently improve the factual expression capability {of FFNs}
by knowledge enhancement and alignment respectively. We first propose
\textsc{K-Dial}, which {explicitly} introduces {extended FFNs in Transformers
to enhance factual knowledge expressions} given the specific patterns of
knowledge-grounded dialogue inputs. Additionally, we apply the reinforcement
learning for factual consistency (RLFC) method to implicitly adjust FFNs'
expressions in responses by aligning with gold knowledge for the factual
consistency preference. To comprehensively assess the factual consistency and
dialogue quality of responses, we employ extensive automatic measures and human
evaluations including sophisticated fine-grained NLI-based metrics.
Experimental results on WoW and CMU\_DoG datasets demonstrate that our methods
efficiently enhance the ability of the FFN module to convey factual knowledge,
validating the efficacy of improving factual consistency for knowledge-grounded
dialogue systems.
Related papers
- Improving the Robustness of Knowledge-Grounded Dialogue via Contrastive
Learning [71.8876256714229]
We propose an entity-based contrastive learning framework for improving the robustness of knowledge-grounded dialogue systems.
Our method achieves new state-of-the-art performance in terms of automatic evaluation scores.
arXiv Detail & Related papers (2024-01-09T05:16:52Z) - RELIC: Investigating Large Language Model Responses using Self-Consistency [58.63436505595177]
Large Language Models (LLMs) are notorious for blending fact with fiction and generating non-factual content, known as hallucinations.
We propose an interactive system that helps users gain insight into the reliability of the generated text.
arXiv Detail & Related papers (2023-11-28T14:55:52Z) - Position Matters! Empirical Study of Order Effect in Knowledge-grounded
Dialogue [54.98184262897166]
We investigate how the order of the knowledge set can influence autoregressive dialogue systems' responses.
We propose a simple and novel technique to alleviate the order effect by modifying the position embeddings of knowledge input.
arXiv Detail & Related papers (2023-02-12T10:13:00Z) - Dynamically Retrieving Knowledge via Query Generation for informative
dialogue response [7.196104022425989]
We design a knowledge-driven dialogue system named DRKQG (emphDynamically Retrieving Knowledge via Query Generation for informative dialogue response)
First, a time-aware mechanism is utilized to capture context information and a query can be generated for retrieving knowledge.
Then, we integrate copy Mechanism and Transformers, which allows the response generation module produces responses derived from the context and retrieved knowledge.
arXiv Detail & Related papers (2022-07-30T03:05:43Z) - RT-KGD: Relation Transition Aware Knowledge-Grounded Dialogue Generation [20.37399983466163]
We propose a Relation Transition aware Knowledge-Grounded Dialogue Generation model (RT-KGD)
Specifically, inspired by the latent logic of human conversation, our model integrates dialogue-level relation transition regularities with turn-level entity semantic information.
In this manner, the interaction between knowledge is considered to produce abundant clues for predicting the appropriate knowledge and generating coherent responses.
arXiv Detail & Related papers (2022-07-17T16:07:38Z) - Kformer: Knowledge Injection in Transformer Feed-Forward Layers [107.71576133833148]
We propose a novel knowledge fusion model, namely Kformer, which incorporates external knowledge through the feed-forward layer in Transformer.
We empirically find that simply injecting knowledge into FFN can facilitate the pre-trained language model's ability and facilitate current knowledge fusion methods.
arXiv Detail & Related papers (2022-01-15T03:00:27Z) - Zero-Resource Knowledge-Grounded Dialogue Generation [29.357221039484568]
We propose representing the knowledge that bridges a context and a response and the way that the knowledge is expressed as latent variables.
We show that our model can achieve comparable performance with state-of-the-art methods that rely on knowledge-grounded dialogues for training.
arXiv Detail & Related papers (2020-08-29T05:48:32Z) - Sequential Latent Knowledge Selection for Knowledge-Grounded Dialogue [51.513276162736844]
We propose a sequential latent variable model as the first approach to this matter.
The model named sequential knowledge transformer (SKT) can keep track of the prior and posterior distribution over knowledge.
arXiv Detail & Related papers (2020-02-18T11:59:59Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.