Attend to the beginning: A study on using bidirectional attention for
extractive summarization
- URL: http://arxiv.org/abs/2002.03405v3
- Date: Sat, 9 May 2020 03:13:38 GMT
- Title: Attend to the beginning: A study on using bidirectional attention for
extractive summarization
- Authors: Ahmed Magooda and Cezary Marcjan
- Abstract summary: We propose attending to the beginning of a document, to improve the performance of extractive summarization models.
We make use of the tendency of introducing important information early in the text, by attending to the first few sentences in generic textual data.
- Score: 1.148539813252112
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Forum discussion data differ in both structure and properties from generic
form of textual data such as news. Henceforth, summarization techniques should,
in turn, make use of such differences, and craft models that can benefit from
the structural nature of discussion data. In this work, we propose attending to
the beginning of a document, to improve the performance of extractive
summarization models when applied to forum discussion data. Evaluations
demonstrated that with the help of bidirectional attention mechanism, attending
to the beginning of a document (initial comment/post) in a discussion thread,
can introduce a consistent boost in ROUGE scores, as well as introducing a new
State Of The Art (SOTA) ROUGE scores on the forum discussions dataset.
Additionally, we explored whether this hypothesis is extendable to other
generic forms of textual data. We make use of the tendency of introducing
important information early in the text, by attending to the first few
sentences in generic textual data. Evaluations demonstrated that attending to
introductory sentences using bidirectional attention, improves the performance
of extractive summarization models when even applied to more generic form of
textual data.
Related papers
- Enhancing Argument Structure Extraction with Efficient Leverage of
Contextual Information [79.06082391992545]
We propose an Efficient Context-aware model (ECASE) that fully exploits contextual information.
We introduce a sequence-attention module and distance-weighted similarity loss to aggregate contextual information and argumentative information.
Our experiments on five datasets from various domains demonstrate that our model achieves state-of-the-art performance.
arXiv Detail & Related papers (2023-10-08T08:47:10Z) - A Data-driven Latent Semantic Analysis for Automatic Text Summarization
using LDA Topic Modelling [0.0]
This study presents the Latent Dirichlet Allocation (LDA) approach used to perform topic modelling.
The visualisation provides an overarching view of the main topics while allowing and attributing deep meaning to the prevalence individual topic.
The results suggest the terms ranked purely by considering their probability of the topic prevalence within the processed document.
arXiv Detail & Related papers (2022-07-23T11:04:03Z) - Dialogue Summarization with Supporting Utterance Flow Modeling and Fact
Regularization [58.965859508695225]
We propose an end-to-end neural model for dialogue summarization with two novel modules.
The supporting utterance flow modeling helps to generate a coherent summary by smoothly shifting the focus from the former utterances to the later ones.
The fact regularization encourages the generated summary to be factually consistent with the ground-truth summary during model training.
arXiv Detail & Related papers (2021-08-03T03:09:25Z) - Topic Modeling Based Extractive Text Summarization [0.0]
We propose a novel method to summarize a text document by clustering its contents based on latent topics.
We utilize the lesser used and challenging WikiHow dataset in our approach to text summarization.
arXiv Detail & Related papers (2021-06-29T12:28:19Z) - ConvoSumm: Conversation Summarization Benchmark and Improved Abstractive
Summarization with Argument Mining [61.82562838486632]
We crowdsource four new datasets on diverse online conversation forms of news comments, discussion forums, community question answering forums, and email threads.
We benchmark state-of-the-art models on our datasets and analyze characteristics associated with the data.
arXiv Detail & Related papers (2021-06-01T22:17:13Z) - Transductive Learning for Abstractive News Summarization [24.03781438153328]
We propose the first application of transductive learning to summarization.
We show that our approach yields state-of-the-art results on CNN/DM and NYT datasets.
arXiv Detail & Related papers (2021-04-17T17:33:12Z) - Weakly-Supervised Aspect-Based Sentiment Analysis via Joint
Aspect-Sentiment Topic Embedding [71.2260967797055]
We propose a weakly-supervised approach for aspect-based sentiment analysis.
We learn sentiment, aspect> joint topic embeddings in the word embedding space.
We then use neural models to generalize the word-level discriminative information.
arXiv Detail & Related papers (2020-10-13T21:33:24Z) - Abstractive Summarization of Spoken and Written Instructions with BERT [66.14755043607776]
We present the first application of the BERTSum model to conversational language.
We generate abstractive summaries of narrated instructional videos across a wide variety of topics.
We envision this integrated as a feature in intelligent virtual assistants, enabling them to summarize both written and spoken instructional content upon request.
arXiv Detail & Related papers (2020-08-21T20:59:34Z) - Extractive Summarization as Text Matching [123.09816729675838]
This paper creates a paradigm shift with regard to the way we build neural extractive summarization systems.
We formulate the extractive summarization task as a semantic text matching problem.
We have driven the state-of-the-art extractive result on CNN/DailyMail to a new level (44.41 in ROUGE-1)
arXiv Detail & Related papers (2020-04-19T08:27:57Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.