PLATO-K: Internal and External Knowledge Enhanced Dialogue Generation

• URL: http://arxiv.org/abs/2211.00910v1
• Date: Wed, 2 Nov 2022 06:23:16 GMT
• Title: PLATO-K: Internal and External Knowledge Enhanced Dialogue Generation
• Authors: Siqi Bao, Huang He, Jun Xu, Hua Lu, Fan Wang, Hua Wu, Han Zhou, Wenquan Wu, Zheng-Yu Niu, Haifeng Wang
• Abstract summary: We introduce PLATO-K based on two-stage dialogic learning to strengthen internal knowledge and external knowledge exploitation. In the first stage, PLATO-K learns through massive dialogue corpora and memorizes essential knowledge into model parameters. In the second stage, PLATO-K mimics human beings to search for external information and to leverage the knowledge in response generation.
• Score: 49.43839526180323
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, the practical deployment of open-domain dialogue systems has been plagued by the knowledge issue of information deficiency and factual inaccuracy. To this end, we introduce PLATO-K based on two-stage dialogic learning to strengthen internal knowledge memorization and external knowledge exploitation. In the first stage, PLATO-K learns through massive dialogue corpora and memorizes essential knowledge into model parameters. In the second stage, PLATO-K mimics human beings to search for external information and to leverage the knowledge in response generation. Extensive experiments reveal that the knowledge issue is alleviated significantly in PLATO-K with such comprehensive internal and external knowledge enhancement. Compared to the existing state-of-the-art Chinese dialogue model, the overall engagingness of PLATO-K is improved remarkably by 36.2% and 49.2% on chit-chat and knowledge-intensive conversations.

Related papers

• Beyond Factuality: A Comprehensive Evaluation of Large Language Models as Knowledge Generators [78.63553017938911]
  Large language models (LLMs) outperform information retrieval techniques for downstream knowledge-intensive tasks. However, community concerns abound regarding the factuality and potential implications of using this uncensored knowledge. We introduce CONNER, designed to evaluate generated knowledge from six important perspectives.
  arXiv  Detail & Related papers  (2023-10-11T08:22:37Z)
• The Knowledge Alignment Problem: Bridging Human and External Knowledge for Large Language Models [65.80573571314534]
  We introduce MixAlign, a framework that interacts with both the human user and the knowledge base to obtain and integrate clarifications on how the user question relates to the stored information. Experimental results highlight the crucial role of knowledge alignment in boosting model performance and mitigating hallucination, with improvements noted up to 22.2% and 27.1% respectively.
  arXiv  Detail & Related papers  (2023-05-23T04:22:50Z)
• KPT: Keyword-guided Pre-training for Grounded Dialog Generation [82.68787152707455]
  We propose KPT (guided Pre-Training), a novel self-supervised pre-training method for grounded dialog generation. Specifically, we use a pre-trained language model to extract the most uncertain tokens in the dialog as keywords. We conduct extensive experiments on various few-shot knowledge-grounded generation tasks, including grounding on dialog acts, knowledge graphs, persona descriptions, and Wikipedia passages.
  arXiv  Detail & Related papers  (2022-12-04T04:05:01Z)
• SINC: Service Information Augmented Open-Domain Conversation [46.912064636311825]
  We propose a knowledge-driven dialogue system using dynamic service information. We release the first open domain Chinese service knowledge dialogue dataset DuSinc. Both automatic evaluation and human evaluation show that our proposed new method can significantly improve the effect of open-domain conversation.
  arXiv  Detail & Related papers  (2022-06-28T13:41:48Z)
• Lexical Knowledge Internalization for Neural Dialog Generation [36.27946635687281]
  We propose knowledge internalization (KI), which aims to complement the lexical knowledge into neural dialog models. To tackle the challenge due to the large scale of lexical knowledge, we adopt the contrastive learning approach and create an effective token-level lexical knowledge retriever.
  arXiv  Detail & Related papers  (2022-05-04T08:23:44Z)
• Open-domain Dialogue Generation Grounded with Dynamic Multi-form Knowledge Fusion [9.45662259790057]
  This paper presents a new dialogue generation model, Dynamic Multi-form Knowledge Fusion based Open-domain Chatt-ing Machine (DMKCM) DMKCM applies an indexed text (a virtual Knowledge Base) to locate relevant documents as 1st hop and then expands the content of the dialogue and its 1st hop using a commonsense knowledge graph to get apposite triples as 2nd hop. Experimental results indicate the effectiveness of our method in terms of dialogue coherence and informativeness.
  arXiv  Detail & Related papers  (2022-04-24T10:32:48Z)
• Knowledge-Grounded Dialogue with Reward-Driven Knowledge Selection [1.1633929083694388]
  Knoformer is a dialogue response generation model based on reinforcement learning. It can automatically select one or more related knowledge from the knowledge pool and does not need knowledge labels during training.
  arXiv  Detail & Related papers  (2021-08-31T08:53:08Z)
• Sequential Latent Knowledge Selection for Knowledge-Grounded Dialogue [51.513276162736844]
  We propose a sequential latent variable model as the first approach to this matter. The model named sequential knowledge transformer (SKT) can keep track of the prior and posterior distribution over knowledge.
  arXiv  Detail & Related papers  (2020-02-18T11:59:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.