Sentence Embeddings as an intermediate target in end-to-end summarisation

• URL: http://arxiv.org/abs/2505.03481v1
• Date: Tue, 06 May 2025 12:34:59 GMT
• Title: Sentence Embeddings as an intermediate target in end-to-end summarisation
• Authors: Maciej Zembrzuski, Saad Mahamood,
• Abstract summary: We show that by combining an extractive approach with externally pre-trained sentence level embeddings we can outperform existing methods.<n>We also prove that predicting sentence level embedding of a summary increases the quality of an end-to-end system for loosely aligned source to target corpora.
• Score: 1.4044612085920334
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Current neural network-based methods to the problem of document summarisation struggle when applied to datasets containing large inputs. In this paper we propose a new approach to the challenge of content-selection when dealing with end-to-end summarisation of user reviews of accommodations. We show that by combining an extractive approach with externally pre-trained sentence level embeddings in an addition to an abstractive summarisation model we can outperform existing methods when this is applied to the task of summarising a large input dataset. We also prove that predicting sentence level embedding of a summary increases the quality of an end-to-end system for loosely aligned source to target corpora, than compared to commonly predicting probability distributions of sentence selection.

Related papers

• Balancing Diversity and Risk in LLM Sampling: How to Select Your Method and Parameter for Open-Ended Text Generation [60.493180081319785]
  We propose a systematic way to estimate the capacity of a truncation sampling method by considering the trade-off between diversity and risk at each decoding step.<n>Our work offers a comprehensive comparison of existing truncation sampling methods and serves as a practical user guideline for their parameter selection.
  arXiv  Detail & Related papers  (2024-08-24T14:14:32Z)
• On Context Utilization in Summarization with Large Language Models [83.84459732796302]
  Large language models (LLMs) excel in abstractive summarization tasks, delivering fluent and pertinent summaries. Recent advancements have extended their capabilities to handle long-input contexts, exceeding 100k tokens. We conduct the first comprehensive study on context utilization and position bias in summarization.
  arXiv  Detail & Related papers  (2023-10-16T16:45:12Z)
• Enhancing Topic Extraction in Recommender Systems with Entropy Regularization [2.7286395031146062]
  This paper introduces a novel approach called entropy regularization to address the issue of low explainability of recommender systems. Experiment results show a significant improvement in topic coherence, which is quantified by cosine similarity on word embeddings.
  arXiv  Detail & Related papers  (2023-06-12T20:05:09Z)
• A New Sentence Extraction Strategy for Unsupervised Extractive Summarization Methods [26.326800624948344]
  We model the task of extractive text summarization methods from the perspective of Information Theory. To improve the feature distribution and to decrease the mutual information of summarization sentences, we propose a new sentence extraction strategy.
  arXiv  Detail & Related papers  (2021-12-06T18:00:02Z)
• The Summary Loop: Learning to Write Abstractive Summaries Without Examples [21.85348918324668]
  This work presents a new approach to unsupervised abstractive summarization based on maximizing a combination of coverage and fluency for a given length constraint. Key terms are masked out of the original document and must be filled in by a coverage model using the current generated summary. When tested on popular news summarization datasets, the method outperforms previous unsupervised methods by more than 2 R-1 points.
  arXiv  Detail & Related papers  (2021-05-11T23:19:46Z)
• Unsupervised Extractive Summarization using Pointwise Mutual Information [5.544401446569243]
  We propose new metrics of relevance and redundancy using pointwise mutual information (PMI) between sentences. We show that our method outperforms similarity-based methods on datasets in a range of domains including news, medical journal articles, and personal anecdotes.
  arXiv  Detail & Related papers  (2021-02-11T21:05:50Z)
• Abstractive Query Focused Summarization with Query-Free Resources [60.468323530248945]
  In this work, we consider the problem of leveraging only generic summarization resources to build an abstractive QFS system. We propose Marge, a Masked ROUGE Regression framework composed of a novel unified representation for summaries and queries. Despite learning from minimal supervision, our system achieves state-of-the-art results in the distantly supervised setting.
  arXiv  Detail & Related papers  (2020-12-29T14:39:35Z)
• Automated Concatenation of Embeddings for Structured Prediction [75.44925576268052]
  We propose Automated Concatenation of Embeddings (ACE) to automate the process of finding better concatenations of embeddings for structured prediction tasks. We follow strategies in reinforcement learning to optimize the parameters of the controller and compute the reward based on the accuracy of a task model.
  arXiv  Detail & Related papers  (2020-10-10T14:03:20Z)
• Summary-Source Proposition-level Alignment: Task, Datasets and Supervised Baseline [94.0601799665342]
  Aligning sentences in a reference summary with their counterparts in source documents was shown as a useful auxiliary summarization task. We propose establishing summary-source alignment as an explicit task, while introducing two major novelties. We create a novel training dataset for proposition-level alignment, derived automatically from available summarization evaluation data. We present a supervised proposition alignment baseline model, showing improved alignment-quality over the unsupervised approach.
  arXiv  Detail & Related papers  (2020-09-01T17:27:12Z)
• SummPip: Unsupervised Multi-Document Summarization with Sentence Graph Compression [61.97200991151141]
  SummPip is an unsupervised method for multi-document summarization. We convert the original documents to a sentence graph, taking both linguistic and deep representation into account. We then apply spectral clustering to obtain multiple clusters of sentences, and finally compress each cluster to generate the final summary.
  arXiv  Detail & Related papers  (2020-07-17T13:01:15Z)
• Extractive Summarization as Text Matching [123.09816729675838]
  This paper creates a paradigm shift with regard to the way we build neural extractive summarization systems. We formulate the extractive summarization task as a semantic text matching problem. We have driven the state-of-the-art extractive result on CNN/DailyMail to a new level (44.41 in ROUGE-1)
  arXiv  Detail & Related papers  (2020-04-19T08:27:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.