Beyond Goldfish Memory: Long-Term Open-Domain Conversation

• URL: http://arxiv.org/abs/2107.07567v1
• Date: Thu, 15 Jul 2021 19:01:08 GMT
• Title: Beyond Goldfish Memory: Long-Term Open-Domain Conversation
• Authors: Jing Xu, Arthur Szlam, Jason Weston
• Abstract summary: We release a human-human dataset consisting of multiple chat sessions. We show how existing models trained on existing datasets perform poorly in this long-term conversation setting. In particular, we find retrieval-augmented methods and methods with an ability to summarize and recall previous conversations.
• Score: 43.37382902468993
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite recent improvements in open-domain dialogue models, state of the art models are trained and evaluated on short conversations with little context. In contrast, the long-term conversation setting has hardly been studied. In this work we collect and release a human-human dataset consisting of multiple chat sessions whereby the speaking partners learn about each other's interests and discuss the things they have learnt from past sessions. We show how existing models trained on existing datasets perform poorly in this long-term conversation setting in both automatic and human evaluations, and we study long-context models that can perform much better. In particular, we find retrieval-augmented methods and methods with an ability to summarize and recall previous conversations outperform the standard encoder-decoder architectures currently considered state of the art.

Related papers

• Evaluating Very Long-Term Conversational Memory of LLM Agents [95.84027826745609]
  We introduce a machine-human pipeline to generate high-quality, very long-term dialogues. We equip each agent with the capability of sharing and reacting to images. The generated conversations are verified and edited by human annotators for long-range consistency.
  arXiv  Detail & Related papers  (2024-02-27T18:42:31Z)
• Mind the Gap Between Conversations for Improved Long-Term Dialogue Generation [21.109006148673846]
  GapChat is a multi-session dialogue dataset in which the time between each session varies. While the dataset is constructed in real-time, progress on events in speakers' lives is simulated in order to create realistic dialogues occurring across a long timespan. We show that time-aware models perform better in metrics that judge the relevance of the chosen topics and the information gained from the conversation.
  arXiv  Detail & Related papers  (2023-10-24T00:12:38Z)
• Conversation Chronicles: Towards Diverse Temporal and Relational Dynamics in Multi-Session Conversations [9.249662593315541]
  We introduce a new 1M multi-session dialogue dataset, Conversation Chronicles, for implementing a long-term conversation setup. We show that dialogue episodes in Conversation Chronicles reflect those properties while maintaining coherent and consistent interactions. We also propose a dialogue model, called ReBot, which consists of chronological summarization and dialogue generation modules.
  arXiv  Detail & Related papers  (2023-10-20T11:06:21Z)
• AutoConv: Automatically Generating Information-seeking Conversations with Large Language Models [74.10293412011455]
  We propose AutoConv for synthetic conversation generation. Specifically, we formulate the conversation generation problem as a language modeling task. We finetune an LLM with a few human conversations to capture the characteristics of the information-seeking process.
  arXiv  Detail & Related papers  (2023-08-12T08:52:40Z)
• Multi-Party Chat: Conversational Agents in Group Settings with Humans and Models [39.80729604768669]
  We evaluate the ability of language models to act as one or more characters in multi-party conversations. We find that our new dataset, MultiLIGHT, can help bring significant improvements in the group setting.
  arXiv  Detail & Related papers  (2023-04-26T21:41:17Z)
• PLACES: Prompting Language Models for Social Conversation Synthesis [103.94325597273316]
  We use a small set of expert-written conversations as in-context examples to synthesize a social conversation dataset using prompting. We perform several thorough evaluations of our synthetic conversations compared to human-collected conversations.
  arXiv  Detail & Related papers  (2023-02-07T05:48:16Z)
• GODEL: Large-Scale Pre-Training for Goal-Directed Dialog [119.1397031992088]
  We introduce GODEL, a large pre-trained language model for dialog. We show that GODEL outperforms state-of-the-art pre-trained dialog models in few-shot fine-tuning setups. A novel feature of our evaluation methodology is the introduction of a notion of utility that assesses the usefulness of responses.
  arXiv  Detail & Related papers  (2022-06-22T18:19:32Z)
• Summary Grounded Conversation Generation [10.470157142861174]
  We show how pre-trained language models can be used to generate entire conversations, given only a summary of a conversation as the input. We also show that the accuracy of conversation summarization can be improved by augmenting a conversation summarization dataset with generated conversations.
  arXiv  Detail & Related papers  (2021-06-07T04:46:31Z)
• KdConv: A Chinese Multi-domain Dialogue Dataset Towards Multi-turn Knowledge-driven Conversation [66.99734491847076]
  We propose a Chinese multi-domain knowledge-driven conversation dataset, KdConv, which grounds the topics in multi-turn conversations to knowledge graphs. Our corpus contains 4.5K conversations from three domains (film, music, and travel), and 86K utterances with an average turn number of 19.0.
  arXiv  Detail & Related papers  (2020-04-08T16:25:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.