Compressive Summarization with Plausibility and Salience Modeling
- URL: http://arxiv.org/abs/2010.07886v1
- Date: Thu, 15 Oct 2020 17:07:10 GMT
- Title: Compressive Summarization with Plausibility and Salience Modeling
- Authors: Shrey Desai and Jiacheng Xu and Greg Durrett
- Abstract summary: We propose to relax the rigid syntactic constraints on candidate spans and instead leave compression decisions to two data-driven criteria: plausibility and salience.
Our method achieves strong in-domain results on benchmark summarization datasets, and human evaluation shows that the plausibility model generally selects for grammatical and factual deletions.
- Score: 54.37665950633147
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Compressive summarization systems typically rely on a crafted set of
syntactic rules to determine what spans of possible summary sentences can be
deleted, then learn a model of what to actually delete by optimizing for
content selection (ROUGE). In this work, we propose to relax the rigid
syntactic constraints on candidate spans and instead leave compression
decisions to two data-driven criteria: plausibility and salience. Deleting a
span is plausible if removing it maintains the grammaticality and factuality of
a sentence, and spans are salient if they contain important information from
the summary. Each of these is judged by a pre-trained Transformer model, and
only deletions that are both plausible and not salient can be applied. When
integrated into a simple extraction-compression pipeline, our method achieves
strong in-domain results on benchmark summarization datasets, and human
evaluation shows that the plausibility model generally selects for grammatical
and factual deletions. Furthermore, the flexibility of our approach allows it
to generalize cross-domain: our system fine-tuned on only 500 samples from a
new domain can match or exceed an in-domain extractive model trained on much
more data.
Related papers
- Entity or Relation Embeddings? An Analysis of Encoding Strategies for Relation Extraction [19.019881161010474]
Relation extraction is essentially a text classification problem, which can be tackled by fine-tuning a pre-trained language model (LM)
Existing approaches therefore solve the problem in an indirect way: they fine-tune an LM to learn embeddings of the head and tail entities, and then predict the relationship from these entity embeddings.
Our hypothesis in this paper is that relation extraction models can be improved by capturing relationships in a more direct way.
arXiv Detail & Related papers (2023-12-18T09:58:19Z) - AMRFact: Enhancing Summarization Factuality Evaluation with AMR-Driven Negative Samples Generation [57.8363998797433]
We propose AMRFact, a framework that generates perturbed summaries using Abstract Meaning Representations (AMRs)
Our approach parses factually consistent summaries into AMR graphs and injects controlled factual inconsistencies to create negative examples, allowing for coherent factually inconsistent summaries to be generated with high error-type coverage.
arXiv Detail & Related papers (2023-11-16T02:56:29Z) - Learning with Rejection for Abstractive Text Summarization [42.15551472507393]
We propose a training objective for abstractive summarization based on rejection learning.
We show that our method considerably improves the factuality of generated summaries in automatic and human evaluations.
arXiv Detail & Related papers (2023-02-16T19:07:08Z) - Improving Factual Consistency in Summarization with Compression-Based
Post-Editing [146.24839415743358]
We show that a model-agnostic way to address this problem is post-editing the generated summaries.
We propose to use sentence-compression data to train the post-editing model to take a summary with extrinsic entity errors marked with special tokens.
We show that this model improves factual consistency while maintaining ROUGE, improving entity precision by up to 30% on XSum, and that this model can be applied on top of another post-editor.
arXiv Detail & Related papers (2022-11-11T13:35:38Z) - BRIO: Bringing Order to Abstractive Summarization [107.97378285293507]
We propose a novel training paradigm which assumes a non-deterministic distribution.
Our method achieves a new state-of-the-art result on the CNN/DailyMail (47.78 ROUGE-1) and XSum (49.07 ROUGE-1) datasets.
arXiv Detail & Related papers (2022-03-31T05:19:38Z) - Multi-Fact Correction in Abstractive Text Summarization [98.27031108197944]
Span-Fact is a suite of two factual correction models that leverages knowledge learned from question answering models to make corrections in system-generated summaries via span selection.
Our models employ single or multi-masking strategies to either iteratively or auto-regressively replace entities in order to ensure semantic consistency w.r.t. the source text.
Experiments show that our models significantly boost the factual consistency of system-generated summaries without sacrificing summary quality in terms of both automatic metrics and human evaluation.
arXiv Detail & Related papers (2020-10-06T02:51:02Z) - Generalized Multi-Output Gaussian Process Censored Regression [7.111443975103331]
We introduce a heteroscedastic multi-output Gaussian process model which combines the non-parametric flexibility of GPs with the ability to leverage information from correlated outputs under input-dependent noise conditions.
Results show how the added flexibility allows our model to better estimate the underlying non-censored (i.e. true) process under potentially complex censoring dynamics.
arXiv Detail & Related papers (2020-09-10T12:46:29Z) - Extractive Summarization as Text Matching [123.09816729675838]
This paper creates a paradigm shift with regard to the way we build neural extractive summarization systems.
We formulate the extractive summarization task as a semantic text matching problem.
We have driven the state-of-the-art extractive result on CNN/DailyMail to a new level (44.41 in ROUGE-1)
arXiv Detail & Related papers (2020-04-19T08:27:57Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.