Meta-Transfer Learning for Low-Resource Abstractive Summarization
- URL: http://arxiv.org/abs/2102.09397v1
- Date: Thu, 18 Feb 2021 14:42:09 GMT
- Title: Meta-Transfer Learning for Low-Resource Abstractive Summarization
- Authors: Yi-Syuan Chen and Hong-Han Shuai
- Abstract summary: Low-Resource Abstractive Summarization aims to leverage past experience to improve the performance with limited labeled examples of target corpus.
We conduct extensive experiments on various summarization corpora with different writing styles and forms.
The results demonstrate that our approach achieves the state-of-the-art on 6 corpora in low-resource scenarios, with only 0.7% of trainable parameters compared to previous work.
- Score: 12.757403709439325
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Neural abstractive summarization has been studied in many pieces of
literature and achieves great success with the aid of large corpora. However,
when encountering novel tasks, one may not always benefit from transfer
learning due to the domain shifting problem, and overfitting could happen
without adequate labeled examples. Furthermore, the annotations of abstractive
summarization are costly, which often demand domain knowledge to ensure the
ground-truth quality. Thus, there are growing appeals for Low-Resource
Abstractive Summarization, which aims to leverage past experience to improve
the performance with limited labeled examples of target corpus. In this paper,
we propose to utilize two knowledge-rich sources to tackle this problem, which
are large pre-trained models and diverse existing corpora. The former can
provide the primary ability to tackle summarization tasks; the latter can help
discover common syntactic or semantic information to improve the generalization
ability. We conduct extensive experiments on various summarization corpora with
different writing styles and forms. The results demonstrate that our approach
achieves the state-of-the-art on 6 corpora in low-resource scenarios, with only
0.7% of trainable parameters compared to previous work.
Related papers
- ACLSum: A New Dataset for Aspect-based Summarization of Scientific
Publications [10.529898520273063]
ACLSum is a novel summarization dataset carefully crafted and evaluated by domain experts.
In contrast to previous datasets, ACLSum facilitates multi-aspect summarization of scientific papers.
arXiv Detail & Related papers (2024-03-08T13:32:01Z) - Factually Consistent Summarization via Reinforcement Learning with
Textual Entailment Feedback [57.816210168909286]
We leverage recent progress on textual entailment models to address this problem for abstractive summarization systems.
We use reinforcement learning with reference-free, textual entailment rewards to optimize for factual consistency.
Our results, according to both automatic metrics and human evaluation, show that our method considerably improves the faithfulness, salience, and conciseness of the generated summaries.
arXiv Detail & Related papers (2023-05-31T21:04:04Z) - Curriculum-Guided Abstractive Summarization [45.57561926145256]
Recent Transformer-based summarization models have provided a promising approach to abstractive summarization.
These models have two shortcomings: (1) they often perform poorly in content selection, and (2) their training strategy is not quite efficient, which restricts model performance.
In this paper, we explore two ways to compensate for these pitfalls. First, we augment the Transformer network with a sentence cross-attention module in the decoder, encouraging more abstraction of salient content.
arXiv Detail & Related papers (2023-02-02T11:09:37Z) - OpineSum: Entailment-based self-training for abstractive opinion
summarization [6.584115526134759]
We present a novel self-training approach, OpineSum, for abstractive opinion summarization.
The summaries in this approach are built using a novel application of textual entailment.
OpineSum achieves state-of-the-art performance in both settings.
arXiv Detail & Related papers (2022-12-21T06:20:28Z) - Summary-Oriented Vision Modeling for Multimodal Abstractive
Summarization [63.320005222549646]
Multimodal abstractive summarization (MAS) aims to produce a concise summary given the multimodal data (text and vision)
We propose to improve the summary quality through summary-oriented visual features.
Experiments on 44 languages, covering mid-high, low-, and zero-resource scenarios, verify the effectiveness and superiority of the proposed approach.
arXiv Detail & Related papers (2022-12-15T09:05:26Z) - Salience Allocation as Guidance for Abstractive Summarization [61.31826412150143]
We propose a novel summarization approach with a flexible and reliable salience guidance, namely SEASON (SaliencE Allocation as Guidance for Abstractive SummarizatiON)
SEASON utilizes the allocation of salience expectation to guide abstractive summarization and adapts well to articles in different abstractiveness.
arXiv Detail & Related papers (2022-10-22T02:13:44Z) - GSum: A General Framework for Guided Neural Abstractive Summarization [102.29593069542976]
We propose a general and guided summarization framework (GSum) that can effectively take different kinds of external guidance as input.
Experiments demonstrate that this model is effective, achieving state-of-the-art performance according to ROUGE on 4 popular summarization datasets.
arXiv Detail & Related papers (2020-10-15T20:46:14Z) - Summarizing Text on Any Aspects: A Knowledge-Informed Weakly-Supervised
Approach [89.56158561087209]
We study summarizing on arbitrary aspects relevant to the document.
Due to the lack of supervision data, we develop a new weak supervision construction method and an aspect modeling scheme.
Experiments show our approach achieves performance boosts on summarizing both real and synthetic documents.
arXiv Detail & Related papers (2020-10-14T03:20:46Z) - What Have We Achieved on Text Summarization? [32.90169694110989]
We aim to gain more understanding of summarization systems with respect to their strengths and limits on a fine-grained syntactic and semantic level.
We quantify 8 major sources of errors on 10 representative summarization models manually.
arXiv Detail & Related papers (2020-10-09T12:39:33Z) - Unsupervised Opinion Summarization with Noising and Denoising [85.49169453434554]
We create a synthetic dataset from a corpus of user reviews by sampling a review, pretending it is a summary, and generating noisy versions thereof.
At test time, the model accepts genuine reviews and generates a summary containing salient opinions, treating those that do not reach consensus as noise.
arXiv Detail & Related papers (2020-04-21T16:54:57Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.