Local Knowledge Powered Conversational Agents
- URL: http://arxiv.org/abs/2010.10150v1
- Date: Tue, 20 Oct 2020 09:34:40 GMT
- Title: Local Knowledge Powered Conversational Agents
- Authors: Sashank Santhanam, Wei Ping, Raul Puri, Mohammad Shoeybi, Mostofa
Patwary, Bryan Catanzaro
- Abstract summary: We propose a dialog framework that incorporates both local knowledge as well as users' past dialogues to generate high quality conversations.
Using our framework and dataset, we demonstrate that incorporating local knowledge can largely improve informativeness, coherency and realisticness measures.
Our model with 8.3B parameters can generate human-like responses as rated by various human evaluations in a single-turn dialog setting.
- Score: 29.966845949225792
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: State-of-the-art conversational agents have advanced significantly in
conjunction with the use of large transformer-based language models. However,
even with these advancements, conversational agents still lack the ability to
produce responses that are informative and coherent with the local context. In
this work, we propose a dialog framework that incorporates both local knowledge
as well as users' past dialogues to generate high quality conversations. We
introduce an approach to build a dataset based on Reddit conversations, where
outbound URL links are widely available in the conversations and the
hyperlinked documents can be naturally included as local external knowledge.
Using our framework and dataset, we demonstrate that incorporating local
knowledge can largely improve informativeness, coherency and realisticness
measures using human evaluations. In particular, our approach consistently
outperforms the state-of-the-art conversational model on the Reddit dataset
across all three measures. We also find that scaling the size of our models
from 117M to 8.3B parameters yields consistent improvement of validation
perplexity as well as human evaluated metrics. Our model with 8.3B parameters
can generate human-like responses as rated by various human evaluations in a
single-turn dialog setting.
Related papers
- $\textit{Dial BeInfo for Faithfulness}$: Improving Factuality of
Information-Seeking Dialogue via Behavioural Fine-Tuning [55.96744451743273]
We introduce BeInfo, a method that applies behavioural tuning to aid information-seeking dialogue systems.
We show that models tuned with BeInfo become considerably more faithful to the knowledge source.
We also show that the models with 3B parameters tuned with BeInfo demonstrate strong performance on data from real production' conversations.
arXiv Detail & Related papers (2023-11-16T11:25:44Z) - CGoDial: A Large-Scale Benchmark for Chinese Goal-oriented Dialog
Evaluation [75.60156479374416]
CGoDial is a new challenging and comprehensive Chinese benchmark for Goal-oriented Dialog evaluation.
It contains 96,763 dialog sessions and 574,949 dialog turns totally, covering three datasets with different knowledge sources.
To bridge the gap between academic benchmarks and spoken dialog scenarios, we either collect data from real conversations or add spoken features to existing datasets via crowd-sourcing.
arXiv Detail & Related papers (2022-11-21T16:21:41Z) - Controllable Dialogue Simulation with In-Context Learning [39.04491297557292]
textscDialogic is a dialogue simulation method based on large language model in-context learning.
Our method can rapidly expand a small set of dialogue data with minimum or zero human involvement.
Our simulated dialogues have near-human fluency and annotation accuracy.
arXiv Detail & Related papers (2022-10-09T06:32:58Z) - Knowledge-Grounded Conversational Data Augmentation with Generative
Conversational Networks [76.11480953550013]
We take a step towards automatically generating conversational data using Generative Conversational Networks.
We evaluate our approach on conversations with and without knowledge on the Topical Chat dataset.
arXiv Detail & Related papers (2022-07-22T22:37:14Z) - Grounding in social media: An approach to building a chit-chat dialogue
model [9.247397520986999]
Building open-domain dialogue systems capable of rich human-like conversational ability is one of the fundamental challenges in language generation.
Current work on knowledge-grounded dialogue generation primarily focuses on persona incorporation or searching a fact-based structured knowledge source such as Wikipedia.
Our method takes a broader and simpler approach, which aims to improve the raw conversation ability of the system by mimicking the human response behavior on social media.
arXiv Detail & Related papers (2022-06-12T09:01:57Z) - What is wrong with you?: Leveraging User Sentiment for Automatic Dialog
Evaluation [73.03318027164605]
We propose to use information that can be automatically extracted from the next user utterance as a proxy to measure the quality of the previous system response.
Our model generalizes across both spoken and written open-domain dialog corpora collected from real and paid users.
arXiv Detail & Related papers (2022-03-25T22:09:52Z) - Training Conversational Agents with Generative Conversational Networks [74.9941330874663]
We use Generative Conversational Networks to automatically generate data and train social conversational agents.
We evaluate our approach on TopicalChat with automatic metrics and human evaluators, showing that with 10% of seed data it performs close to the baseline that uses 100% of the data.
arXiv Detail & Related papers (2021-10-15T21:46:39Z) - Evaluating Groundedness in Dialogue Systems: The BEGIN Benchmark [29.722504033424382]
Knowledge-grounded dialogue agents are systems designed to conduct a conversation based on externally provided background information, such as a Wikipedia page.
We introduce the Benchmark for Evaluation of Grounded INteraction (BEGIN)
BEGIN consists of 8113 dialogue turns generated by language-model-based dialogue systems, accompanied by humans annotations specifying the relationship between the system's response and the background information.
arXiv Detail & Related papers (2021-04-30T20:17:52Z) - A Few-Shot Semantic Parser for Wizard-of-Oz Dialogues with the Precise
ThingTalk Representation [5.56536459714557]
Previous attempts to build effective semantics for Wizard-of-Oz (WOZ) conversations suffer from the difficulty in acquiring a high-quality, manually annotated training set.
This paper proposes a new dialogue representation and a sample-efficient methodology that can predict precise dialogue states in WOZ conversations.
arXiv Detail & Related papers (2020-09-16T22:52:46Z) - Learning an Unreferenced Metric for Online Dialogue Evaluation [53.38078951628143]
We propose an unreferenced automated evaluation metric that uses large pre-trained language models to extract latent representations of utterances.
We show that our model achieves higher correlation with human annotations in an online setting, while not requiring true responses for comparison during inference.
arXiv Detail & Related papers (2020-05-01T20:01:39Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.