A Focused Study on Sequence Length for Dialogue Summarization

• URL: http://arxiv.org/abs/2209.11910v1
• Date: Sat, 24 Sep 2022 02:49:48 GMT
• Title: A Focused Study on Sequence Length for Dialogue Summarization
• Authors: Bin Wang, Chen Zhang, Chengwei Wei, Haizhou Li
• Abstract summary: We analyze the length differences between existing models' outputs and the corresponding human references. We identify salient features for summary length prediction by comparing different model settings. Third, we experiment with a length-aware summarizer and show notable improvement on existing models if summary length can be well incorporated.
• Score: 68.73335643440957
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Output length is critical to dialogue summarization systems. The dialogue summary length is determined by multiple factors, including dialogue complexity, summary objective, and personal preferences. In this work, we approach dialogue summary length from three perspectives. First, we analyze the length differences between existing models' outputs and the corresponding human references and find that summarization models tend to produce more verbose summaries due to their pretraining objectives. Second, we identify salient features for summary length prediction by comparing different model settings. Third, we experiment with a length-aware summarizer and show notable improvement on existing models if summary length can be well incorporated. Analysis and experiments are conducted on popular DialogSum and SAMSum datasets to validate our findings.

Related papers

• Increasing faithfulness in human-human dialog summarization with Spoken Language Understanding tasks [0.0]
  We propose an exploration of how incorporating task-related information can enhance the summarization process. Results show that integrating models with task-related information improves summary accuracy, even with varying word error rates.
  arXiv  Detail & Related papers  (2024-09-16T08:15:35Z)
• A Modular Approach for Multimodal Summarization of TV Shows [55.20132267309382]
  We present a modular approach where separate components perform specialized sub-tasks. Our modules involve detecting scene boundaries, reordering scenes so as to minimize the number of cuts between different events, converting visual information to text, summarizing the dialogue in each scene, and fusing the scene summaries into a final summary for the entire episode. We also present a new metric, PRISMA, to measure both precision and recall of generated summaries, which we decompose into atomic facts.
  arXiv  Detail & Related papers  (2024-03-06T16:10:01Z)
• Long Dialog Summarization: An Analysis [28.223798877781054]
  This work emphasizes the significance of creating coherent and contextually rich summaries for effective communication in various applications. We explore current state-of-the-art approaches for long dialog summarization in different domains and benchmark metrics based evaluations show that one single model does not perform well across various areas for distinct summarization tasks.
  arXiv  Detail & Related papers  (2024-02-26T19:35:45Z)
• SPECTRUM: Speaker-Enhanced Pre-Training for Long Dialogue Summarization [48.284512017469524]
  Multi-turn dialogues are characterized by their extended length and the presence of turn-taking conversations. Traditional language models often overlook the distinct features of these dialogues by treating them as regular text. We propose a speaker-enhanced pre-training method for long dialogue summarization.
  arXiv  Detail & Related papers  (2024-01-31T04:50:00Z)
• Instructive Dialogue Summarization with Query Aggregations [41.89962538701501]
  We introduce instruction-finetuned language models to expand the capability set of dialogue summarization models. We propose a three-step approach to synthesize high-quality query-based summarization triples. By training a unified model called InstructDS on three summarization datasets with multi-purpose instructive triples, we expand the capability of dialogue summarization models.
  arXiv  Detail & Related papers  (2023-10-17T04:03:00Z)
• DIONYSUS: A Pre-trained Model for Low-Resource Dialogue Summarization [127.714919036388]
  DIONYSUS is a pre-trained encoder-decoder model for summarizing dialogues in any new domain. Our experiments show that DIONYSUS outperforms existing methods on six datasets.
  arXiv  Detail & Related papers  (2022-12-20T06:21:21Z)
• Analyzing and Evaluating Faithfulness in Dialogue Summarization [67.07947198421421]
  We first perform the fine-grained human analysis on the faithfulness of dialogue summaries and observe that over 35% of generated summaries are faithfully inconsistent respective the source dialogues. We present a new model-level faithfulness evaluation method. It examines generation models with multi-choice questions created by rule-based transformations.
  arXiv  Detail & Related papers  (2022-10-21T07:22:43Z)
• An Exploratory Study on Long Dialogue Summarization: What Works and What's Next [33.1899354772074]
  We study long dialogue summarization by investigating three strategies to deal with the lengthy input problem and locate relevant information. Our experimental results on three long dialogue datasets (QMSum, MediaSum, SummScreen) show that the retrieve-then-summarize pipeline models yield the best performance.
  arXiv  Detail & Related papers  (2021-09-10T01:38:26Z)
• Controllable Abstractive Dialogue Summarization with Sketch Supervision [56.59357883827276]
  Our model achieves state-of-the-art performance on the largest dialogue summarization corpus SAMSum, with as high as 50.79 in ROUGE-L score.
  arXiv  Detail & Related papers  (2021-05-28T19:05:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.