Neural Conversation Models and How to Rein Them in: A Survey of Failures and Fixes

• URL: http://arxiv.org/abs/2308.06095v1
• Date: Fri, 11 Aug 2023 12:07:45 GMT
• Title: Neural Conversation Models and How to Rein Them in: A Survey of Failures and Fixes
• Authors: Fabian Galetzka, Anne Beyer, David Schlangen
• Abstract summary: Recent conditional language models are able to continue any kind of text source in an often seemingly fluent way. From a linguistic perspective, contributing to a conversation is high. Recent approaches try to tame the underlying language models at various intervention points.
• Score: 17.489075240435348
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent conditional language models are able to continue any kind of text source in an often seemingly fluent way. This fact encouraged research in the area of open-domain conversational systems that are based on powerful language models and aim to imitate an interlocutor by generating appropriate contributions to a written dialogue. From a linguistic perspective, however, the complexity of contributing to a conversation is high. In this survey, we interpret Grice's maxims of cooperative conversation from the perspective of this specific research area and systematize the literature under the aspect of what makes a contribution appropriate: A neural conversation model has to be fluent, informative, consistent, coherent, and follow social norms. In order to ensure these qualities, recent approaches try to tame the underlying language models at various intervention points, such as data, training regime or decoding. Sorted by these categories and intervention points, we discuss promising attempts and suggest novel ways for future research.

Related papers

• WavChat: A Survey of Spoken Dialogue Models [66.82775211793547]
  Recent advancements in spoken dialogue models, exemplified by systems like GPT-4o, have captured significant attention in the speech domain. These advanced spoken dialogue models not only comprehend audio, music, and other speech-related features, but also capture stylistic and timbral characteristics in speech. Despite the progress in spoken dialogue systems, there is a lack of comprehensive surveys that systematically organize and analyze these systems.
  arXiv  Detail & Related papers  (2024-11-15T04:16:45Z)
• Dialogue Quality and Emotion Annotations for Customer Support Conversations [7.218791626731783]
  This paper presents a holistic annotation approach for emotion and conversational quality in the context of bilingual customer support conversations. It provides a unique and valuable resource for the development of text classification models.
  arXiv  Detail & Related papers  (2023-11-23T10:56:14Z)
• Multi-turn Dialogue Comprehension from a Topic-aware Perspective [70.37126956655985]
  This paper proposes to model multi-turn dialogues from a topic-aware perspective. We use a dialogue segmentation algorithm to split a dialogue passage into topic-concentrated fragments in an unsupervised way. We also present a novel model, Topic-Aware Dual-Attention Matching (TADAM) Network, which takes topic segments as processing elements.
  arXiv  Detail & Related papers  (2023-09-18T11:03:55Z)
• Feature Interactions Reveal Linguistic Structure in Language Models [2.0178765779788495]
  We study feature interactions in the context of feature attribution methods for post-hoc interpretability. We work out a grey box methodology, in which we train models to perfection on a formal language classification task. We show that under specific configurations, some methods are indeed able to uncover the grammatical rules acquired by a model.
  arXiv  Detail & Related papers  (2023-06-21T11:24:41Z)
• Advances in Multi-turn Dialogue Comprehension: A Survey [51.215629336320305]
  Training machines to understand natural language and interact with humans is an elusive and essential task of artificial intelligence. This paper reviews the previous methods from the technical perspective of dialogue modeling for the dialogue comprehension task. In addition, we categorize dialogue-related pre-training techniques which are employed to enhance PrLMs in dialogue scenarios.
  arXiv  Detail & Related papers  (2021-10-11T03:52:37Z)
• "How Robust r u?": Evaluating Task-Oriented Dialogue Systems on Spoken Conversations [87.95711406978157]
  This work presents a new benchmark on spoken task-oriented conversations. We study multi-domain dialogue state tracking and knowledge-grounded dialogue modeling. Our data set enables speech-based benchmarking of task-oriented dialogue systems.
  arXiv  Detail & Related papers  (2021-09-28T04:51:04Z)
• Continuity of Topic, Interaction, and Query: Learning to Quote in Online Conversations [23.214585012203084]
  This work studies automatic quotation generation in an online conversation. An encoder-decoder neural framework is employed to continue the context with a quotation. Experiment results on two large-scale datasets in English and Chinese.
  arXiv  Detail & Related papers  (2021-06-18T03:38:48Z)
• Advances in Multi-turn Dialogue Comprehension: A Survey [51.215629336320305]
  We review the previous methods from the perspective of dialogue modeling. We discuss three typical patterns of dialogue modeling that are widely-used in dialogue comprehension tasks.
  arXiv  Detail & Related papers  (2021-03-04T15:50:17Z)
• Probing Task-Oriented Dialogue Representation from Language Models [106.02947285212132]
  This paper investigates pre-trained language models to find out which model intrinsically carries the most informative representation for task-oriented dialogue tasks. We fine-tune a feed-forward layer as the classifier probe on top of a fixed pre-trained language model with annotated labels in a supervised way.
  arXiv  Detail & Related papers  (2020-10-26T21:34:39Z)
• Explainable and Discourse Topic-aware Neural Language Understanding [22.443597046878086]
  Marrying topic models and language models exposes language understanding to a broader source of document-level context beyond sentences. Existing approaches incorporate latent document topic proportions and ignore topical discourse in sentences of the document. We present a novel neural composite language model that exploits both the latent and explainable topics along with topical discourse at sentence-level.
  arXiv  Detail & Related papers  (2020-06-18T15:53:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.