Increasing faithfulness in human-human dialog summarization with Spoken Language Understanding tasks

• URL: http://arxiv.org/abs/2409.10070v1
• Date: Mon, 16 Sep 2024 08:15:35 GMT
• Title: Increasing faithfulness in human-human dialog summarization with Spoken Language Understanding tasks
• Authors: Eunice Akani, Benoit Favre, Frederic Bechet, Romain Gemignani,
• Abstract summary: We propose an exploration of how incorporating task-related information can enhance the summarization process. Results show that integrating models with task-related information improves summary accuracy, even with varying word error rates.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Dialogue summarization aims to provide a concise and coherent summary of conversations between multiple speakers. While recent advancements in language models have enhanced this process, summarizing dialogues accurately and faithfully remains challenging due to the need to understand speaker interactions and capture relevant information. Indeed, abstractive models used for dialog summarization may generate summaries that contain inconsistencies. We suggest using the semantic information proposed for performing Spoken Language Understanding (SLU) in human-machine dialogue systems for goal-oriented human-human dialogues to obtain a more semantically faithful summary regarding the task. This study introduces three key contributions: First, we propose an exploration of how incorporating task-related information can enhance the summarization process, leading to more semantically accurate summaries. Then, we introduce a new evaluation criterion based on task semantics. Finally, we propose a new dataset version with increased annotated data standardized for research on task-oriented dialogue summarization. The study evaluates these methods using the DECODA corpus, a collection of French spoken dialogues from a call center. Results show that integrating models with task-related information improves summary accuracy, even with varying word error rates.

Related papers

• Instructive Dialogue Summarization with Query Aggregations [41.89962538701501]
  We introduce instruction-finetuned language models to expand the capability set of dialogue summarization models. We propose a three-step approach to synthesize high-quality query-based summarization triples. By training a unified model called InstructDS on three summarization datasets with multi-purpose instructive triples, we expand the capability of dialogue summarization models.
  arXiv  Detail & Related papers  (2023-10-17T04:03:00Z)
• DIONYSUS: A Pre-trained Model for Low-Resource Dialogue Summarization [127.714919036388]
  DIONYSUS is a pre-trained encoder-decoder model for summarizing dialogues in any new domain. Our experiments show that DIONYSUS outperforms existing methods on six datasets.
  arXiv  Detail & Related papers  (2022-12-20T06:21:21Z)
• Analyzing and Evaluating Faithfulness in Dialogue Summarization [67.07947198421421]
  We first perform the fine-grained human analysis on the faithfulness of dialogue summaries and observe that over 35% of generated summaries are faithfully inconsistent respective the source dialogues. We present a new model-level faithfulness evaluation method. It examines generation models with multi-choice questions created by rule-based transformations.
  arXiv  Detail & Related papers  (2022-10-21T07:22:43Z)
• Enhancing Semantic Understanding with Self-supervised Methods for Abstractive Dialogue Summarization [4.226093500082746]
  We introduce self-supervised methods to compensate shortcomings to train a dialogue summarization model. Our principle is to detect incoherent information flows using pretext dialogue text to enhance BERT's ability to contextualize the dialogue text representations.
  arXiv  Detail & Related papers  (2022-09-01T07:51:46Z)
• End-to-end Spoken Conversational Question Answering: Task, Dataset and Model [92.18621726802726]
  In spoken question answering, the systems are designed to answer questions from contiguous text spans within the related speech transcripts. We propose a new Spoken Conversational Question Answering task (SCQA), aiming at enabling the systems to model complex dialogue flows. Our main objective is to build the system to deal with conversational questions based on the audio recordings, and to explore the plausibility of providing more cues from different modalities with systems in information gathering.
  arXiv  Detail & Related papers  (2022-04-29T17:56:59Z)
• TODSum: Task-Oriented Dialogue Summarization with State Tracking [16.87549093925514]
  We introduce a large-scale public Task-Oriented Dialogue Summarization dataset, TODSum. Compared to existing work, TODSum suffers from severe scattered information issues and requires strict factual consistency. We propose a state-aware structured dialogue summarization model to integrate dialogue state information and dialogue history.
  arXiv  Detail & Related papers  (2021-10-25T06:53:11Z)
• Who says like a style of Vitamin: Towards Syntax-Aware DialogueSummarization using Multi-task Learning [2.251583286448503]
  We focus on the association between utterances from individual speakers and unique syntactic structures. Speakers have unique textual styles that can contain linguistic information, such as voiceprint. We employ multi-task learning of both syntax-aware information and dialogue summarization.
  arXiv  Detail & Related papers  (2021-09-29T05:30:39Z)
• Topic-Aware Contrastive Learning for Abstractive Dialogue Summarization [41.75442239197745]
  This work proposes two topic-aware contrastive learning objectives, namely coherence detection and sub-summary generation objectives. Experiments on benchmark datasets demonstrate that the proposed simple method significantly outperforms strong baselines.
  arXiv  Detail & Related papers  (2021-09-10T17:03:25Z)
• Topic-Oriented Spoken Dialogue Summarization for Customer Service with Saliency-Aware Topic Modeling [61.67321200994117]
  In a customer service system, dialogue summarization can boost service efficiency by creating summaries for long spoken dialogues. In this work, we focus on topic-oriented dialogue summarization, which generates highly abstractive summaries. We propose a novel topic-augmented two-stage dialogue summarizer ( TDS) jointly with a saliency-aware neural topic model (SATM) for topic-oriented summarization of customer service dialogues.
  arXiv  Detail & Related papers  (2020-12-14T07:50:25Z)
• Multi-View Sequence-to-Sequence Models with Conversational Structure for Abstractive Dialogue Summarization [72.54873655114844]
  Text summarization is one of the most challenging and interesting problems in NLP. This work proposes a multi-view sequence-to-sequence model by first extracting conversational structures of unstructured daily chats from different views to represent conversations. Experiments on a large-scale dialogue summarization corpus demonstrated that our methods significantly outperformed previous state-of-the-art models via both automatic evaluations and human judgment.
  arXiv  Detail & Related papers  (2020-10-04T20:12:44Z)
• Dialogue-Based Relation Extraction [53.2896545819799]
  We present the first human-annotated dialogue-based relation extraction (RE) dataset DialogRE. We argue that speaker-related information plays a critical role in the proposed task, based on an analysis of similarities and differences between dialogue-based and traditional RE tasks. Experimental results demonstrate that a speaker-aware extension on the best-performing model leads to gains in both the standard and conversational evaluation settings.
  arXiv  Detail & Related papers  (2020-04-17T03:51:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.