Knowledge Injection into Dialogue Generation via Language Models
- URL: http://arxiv.org/abs/2004.14614v2
- Date: Mon, 5 Apr 2021 22:50:48 GMT
- Title: Knowledge Injection into Dialogue Generation via Language Models
- Authors: Yi-Lin Tuan, Wei Wei, William Yang Wang
- Abstract summary: InjK is a two-stage approach to inject knowledge into a dialogue generation model.
First, we train a large-scale language model and query it as textual knowledge.
Second, we frame a dialogue generation model to sequentially generate textual knowledge and a corresponding response.
- Score: 85.65843021510521
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Dialogue generation has been successfully learned from scratch by neural
networks, but tends to produce the same general response, e.g., "what are you
talking about?", in many conversations. To reduce this homogeneity, external
knowledge such as the speaker's profile and domain knowledge is applied as an
additional condition to diversify a model's output. The required knowledge to
develop an effective conversation, however, is not always available, which is
different from prior work's assumption that a model always has acquired
sufficient knowledge before chatting. This problem can be detrimental when
applying a dialogue model like this chatting online with unconstrained people
and topics, because the model does not have the needed knowledge. To address
this problem, we propose InjK, which is a two-stage approach to inject
knowledge into a dialogue generation model. First, we train a large-scale
language model and query it as textual knowledge. Second, we frame a dialogue
generation model to sequentially generate textual knowledge and a corresponding
response. Empirically, when a dialogue generation model can only access limited
knowledge, our method outperforms prior work by producing more coherent and
informative responses.
Related papers
- ChatPLUG: Open-Domain Generative Dialogue System with Internet-Augmented
Instruction Tuning for Digital Human [76.62897301298699]
ChatPLUG is a Chinese open-domain dialogue system for digital human applications that instruction finetunes on a wide range of dialogue tasks in a unified internet-augmented format.
We show that modelname outperforms state-of-the-art Chinese dialogue systems on both automatic and human evaluation.
We deploy modelname to real-world applications such as Smart Speaker and Instant Message applications with fast inference.
arXiv Detail & Related papers (2023-04-16T18:16:35Z) - PK-Chat: Pointer Network Guided Knowledge Driven Generative Dialogue
Model [79.64376762489164]
PK-Chat is a Pointer network guided generative dialogue model, incorporating a unified pretrained language model and a pointer network over knowledge graphs.
The words generated by PK-Chat in the dialogue are derived from the prediction of word lists and the direct prediction of the external knowledge graph knowledge.
Based on the PK-Chat, a dialogue system is built for academic scenarios in the case of geosciences.
arXiv Detail & Related papers (2023-04-02T18:23:13Z) - KPT: Keyword-guided Pre-training for Grounded Dialog Generation [82.68787152707455]
We propose KPT (guided Pre-Training), a novel self-supervised pre-training method for grounded dialog generation.
Specifically, we use a pre-trained language model to extract the most uncertain tokens in the dialog as keywords.
We conduct extensive experiments on various few-shot knowledge-grounded generation tasks, including grounding on dialog acts, knowledge graphs, persona descriptions, and Wikipedia passages.
arXiv Detail & Related papers (2022-12-04T04:05:01Z) - Reason first, then respond: Modular Generation for Knowledge-infused
Dialogue [43.64093692715295]
Large language models can produce fluent dialogue but often hallucinate factual inaccuracies.
We propose a modular model, Knowledge to Response, for incorporating knowledge into conversational agents.
In detailed experiments, we find that such a model hallucinates less in knowledge-grounded dialogue tasks.
arXiv Detail & Related papers (2021-11-09T15:29:43Z) - Response Generation with Context-Aware Prompt Learning [19.340498579331555]
We present a novel approach for pre-trained dialogue modeling that casts the dialogue generation problem as a prompt-learning task.
Instead of fine-tuning on limited dialogue data, our approach, DialogPrompt, learns continuous prompt embeddings optimized for dialogue contexts.
Our approach significantly outperforms the fine-tuning baseline and the generic prompt-learning methods.
arXiv Detail & Related papers (2021-11-04T05:40:13Z) - Are Pre-trained Language Models Knowledgeable to Ground Open Domain
Dialogues? [20.598241369838668]
We study knowledge-grounded dialogue generation with pre-trained language models.
We find that by fine-tuning with a few dialogues containing knowledge, the pre-trained language models can outperform the state-of-the-art model.
arXiv Detail & Related papers (2020-11-19T08:22:49Z) - The Adapter-Bot: All-In-One Controllable Conversational Model [66.48164003532484]
We propose a dialogue model that uses a fixed backbone model such as DialGPT and triggers on-demand dialogue skills via different adapters.
Depending on the skills, the model is able to process multiple knowledge types, such as text, tables, and emphatic responses.
We evaluate our model using automatic evaluation by comparing it with existing state-of-the-art conversational models.
arXiv Detail & Related papers (2020-08-28T10:59:31Z) - Sequential Latent Knowledge Selection for Knowledge-Grounded Dialogue [51.513276162736844]
We propose a sequential latent variable model as the first approach to this matter.
The model named sequential knowledge transformer (SKT) can keep track of the prior and posterior distribution over knowledge.
arXiv Detail & Related papers (2020-02-18T11:59:59Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.