Heuristic-based Inter-training to Improve Few-shot Multi-perspective Dialog Summarization

• URL: http://arxiv.org/abs/2203.15590v2
• Date: Wed, 30 Mar 2022 13:40:39 GMT
• Title: Heuristic-based Inter-training to Improve Few-shot Multi-perspective Dialog Summarization
• Authors: Benjamin Sznajder, Chulaka Gunasekara, Guy Lev, Sachin Joshi, Eyal Shnarch, Noam Slonim
• Abstract summary: We study the multi-perspective summarization of customer-care conversations between support agents and customers. We show that our approach supports models to generate multi-perspective summaries with a very small amount of annotated data.
• Score: 13.117715760754077
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Many organizations require their customer-care agents to manually summarize their conversations with customers. These summaries are vital for decision making purposes of the organizations. The perspective of the summary that is required to be created depends on the application of the summaries. With this work, we study the multi-perspective summarization of customer-care conversations between support agents and customers. We observe that there are different heuristics that are associated with summaries of different perspectives, and explore these heuristics to create weak-labeled data for intermediate training of the models before fine-tuning with scarce human annotated summaries. Most importantly, we show that our approach supports models to generate multi-perspective summaries with a very small amount of annotated data. For example, our approach achieves 94\% of the performance (Rouge-2) of a model trained with the original data, by training only with 7\% of the original data.

Related papers

• Tell me what I need to know: Exploring LLM-based (Personalized) Abstractive Multi-Source Meeting Summarization [5.979778557940213]
  Meeting summarization is crucial in digital communication, but existing solutions struggle with salience identification. Previous attempts to address these issues by considering related supplementary resources (e.g., presentation slides) alongside transcripts are hindered by models' limited context sizes. This work explores multi-source meeting summarization considering supplementary materials through a three-stage large language model approach.
  arXiv  Detail & Related papers  (2024-10-18T15:40:48Z)
• Personalized Video Summarization using Text-Based Queries and Conditional Modeling [3.4447129363520337]
  This thesis explores enhancing video summarization by integrating text-based queries and conditional modeling. Evaluation metrics such as accuracy and F1-score assess the quality of the generated summaries.
  arXiv  Detail & Related papers  (2024-08-27T02:43:40Z)
• DataNarrative: Automated Data-Driven Storytelling with Visualizations and Texts [27.218934418961197]
  We introduce a novel task for data story generation and a benchmark containing 1,449 stories from diverse sources. To address the challenges of crafting coherent data stories, we propose a multiagent framework employing two LLM agents. While our agentic framework generally outperforms non-agentic counterparts in both model-based and human evaluations, the results also reveal unique challenges in data story generation.
  arXiv  Detail & Related papers  (2024-08-09T21:31:33Z)
• Information-Theoretic Distillation for Reference-less Summarization [67.51150817011617]
  We present a novel framework to distill a powerful summarizer based on the information-theoretic objective for summarization. We start off from Pythia-2.8B as the teacher model, which is not yet capable of summarization. We arrive at a compact but powerful summarizer with only 568M parameters that performs competitively against ChatGPT.
  arXiv  Detail & Related papers  (2024-03-20T17:42:08Z)
• Sample Efficient Multimodal Semantic Augmentation for Incremental Summarization [13.529904498331673]
  We develop a prompting approach for incremental summarization of task videos. We leverage an existing model for extracting semantic concepts from images. We show the results on a relevant dataset and discuss possible directions for the work.
  arXiv  Detail & Related papers  (2023-03-08T03:58:06Z)
• Summary-Oriented Vision Modeling for Multimodal Abstractive Summarization [63.320005222549646]
  Multimodal abstractive summarization (MAS) aims to produce a concise summary given the multimodal data (text and vision) We propose to improve the summary quality through summary-oriented visual features. Experiments on 44 languages, covering mid-high, low-, and zero-resource scenarios, verify the effectiveness and superiority of the proposed approach.
  arXiv  Detail & Related papers  (2022-12-15T09:05:26Z)
• UniSumm and SummZoo: Unified Model and Diverse Benchmark for Few-Shot Summarization [54.59104881168188]
  textscUniSumm is a unified few-shot summarization model pre-trained with multiple summarization tasks. textscSummZoo is a new benchmark to better evaluate few-shot summarizers.
  arXiv  Detail & Related papers  (2022-11-17T18:54:47Z)
• A Focused Study on Sequence Length for Dialogue Summarization [68.73335643440957]
  We analyze the length differences between existing models' outputs and the corresponding human references. We identify salient features for summary length prediction by comparing different model settings. Third, we experiment with a length-aware summarizer and show notable improvement on existing models if summary length can be well incorporated.
  arXiv  Detail & Related papers  (2022-09-24T02:49:48Z)
• Leveraging Pretrained Models for Automatic Summarization of Doctor-Patient Conversations [9.184616102949228]
  We show that fluent and adequate summaries can be generated with limited training data by fine-tuning BART. Using a carefully chosen fine-tuning dataset, this method is shown to be effective at handling longer conversations.
  arXiv  Detail & Related papers  (2021-09-24T20:18:59Z)
• Dialogue Summarization with Supporting Utterance Flow Modeling and Fact Regularization [58.965859508695225]
  We propose an end-to-end neural model for dialogue summarization with two novel modules. The supporting utterance flow modeling helps to generate a coherent summary by smoothly shifting the focus from the former utterances to the later ones. The fact regularization encourages the generated summary to be factually consistent with the ground-truth summary during model training.
  arXiv  Detail & Related papers  (2021-08-03T03:09:25Z)
• ConvoSumm: Conversation Summarization Benchmark and Improved Abstractive Summarization with Argument Mining [61.82562838486632]
  We crowdsource four new datasets on diverse online conversation forms of news comments, discussion forums, community question answering forums, and email threads. We benchmark state-of-the-art models on our datasets and analyze characteristics associated with the data.
  arXiv  Detail & Related papers  (2021-06-01T22:17:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.