SKT5SciSumm - A Hybrid Generative Approach for Multi-Document Scientific
Summarization
- URL: http://arxiv.org/abs/2402.17311v1
- Date: Tue, 27 Feb 2024 08:33:31 GMT
- Title: SKT5SciSumm - A Hybrid Generative Approach for Multi-Document Scientific
Summarization
- Authors: Huy Quoc To, Hung-Nghiep Tran, Andr'e Greiner-Petter, Felix Beierle,
Akiko Aizawa
- Abstract summary: We propose SKT5SciSumm - a hybrid framework for multi-document scientific summarization (MDSS)
We leverage the Sentence-Transformer version of Scientific Paper Embeddings using Citation-Informed Transformers (SPECTER) to encode and represent textual sentences.
We employ the T5 family of models to generate abstractive summaries using extracted sentences.
- Score: 24.706753105042463
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Summarization for scientific text has shown significant benefits both for the
research community and human society. Given the fact that the nature of
scientific text is distinctive and the input of the multi-document
summarization task is substantially long, the task requires sufficient
embedding generation and text truncation without losing important information.
To tackle these issues, in this paper, we propose SKT5SciSumm - a hybrid
framework for multi-document scientific summarization (MDSS). We leverage the
Sentence-Transformer version of Scientific Paper Embeddings using
Citation-Informed Transformers (SPECTER) to encode and represent textual
sentences, allowing for efficient extractive summarization using k-means
clustering. We employ the T5 family of models to generate abstractive summaries
using extracted sentences. SKT5SciSumm achieves state-of-the-art performance on
the Multi-XScience dataset. Through extensive experiments and evaluation, we
showcase the benefits of our model by using less complicated models to achieve
remarkable results, thereby highlighting its potential in advancing the field
of multi-document summarization for scientific text.
Related papers
- Towards Enhancing Coherence in Extractive Summarization: Dataset and Experiments with LLMs [70.15262704746378]
We propose a systematically created human-annotated dataset consisting of coherent summaries for five publicly available datasets and natural language user feedback.
Preliminary experiments with Falcon-40B and Llama-2-13B show significant performance improvements (10% Rouge-L) in terms of producing coherent summaries.
arXiv Detail & Related papers (2024-07-05T20:25:04Z) - MASSW: A New Dataset and Benchmark Tasks for AI-Assisted Scientific Workflows [58.56005277371235]
We introduce MASSW, a comprehensive text dataset on Multi-Aspect Summarization of ScientificAspects.
MASSW includes more than 152,000 peer-reviewed publications from 17 leading computer science conferences spanning the past 50 years.
We demonstrate the utility of MASSW through multiple novel machine-learning tasks that can be benchmarked using this new dataset.
arXiv Detail & Related papers (2024-06-10T15:19:09Z) - Fusing Multimodal Signals on Hyper-complex Space for Extreme Abstractive
Text Summarization (TL;DR) of Scientific Contents [26.32569293387399]
We deal with a novel task of extreme abstractive text summarization (aka TL;DR generation) by leveraging multiple input modalities.
The mTLDR dataset accompanies a total of 4,182 instances collected from various academic conference proceedings.
We present mTLDRgen, an encoder-decoder-based model that employs a novel dual-fused hyper-complex Transformer.
arXiv Detail & Related papers (2023-06-24T13:51:42Z) - MIReAD: Simple Method for Learning High-quality Representations from
Scientific Documents [77.34726150561087]
We propose MIReAD, a simple method that learns high-quality representations of scientific papers.
We train MIReAD on more than 500,000 PubMed and arXiv abstracts across over 2,000 journal classes.
arXiv Detail & Related papers (2023-05-07T03:29:55Z) - LBMT team at VLSP2022-Abmusu: Hybrid method with text correlation and
generative models for Vietnamese multi-document summarization [1.4716144941085147]
This paper proposes a method for multi-document summarization based on cluster similarity.
After generating summaries by selecting the most important sentences from each cluster, we apply BARTpho and ViT5 to construct the abstractive models.
arXiv Detail & Related papers (2023-04-11T13:15:24Z) - Keyword Extraction from Short Texts with~a~Text-To-Text Transfer
Transformer [0.0]
The paper explores the relevance of the Text-To-Text Transfer Transformer language model (T5) for Polish to the task of intrinsic and extrinsic keyword extraction from short text passages.
We compare the results obtained by four different methods, i.e. plT5kw, extremeText, TermoPL, KeyBERT and conclude that the plT5kw model yields particularly promising results for both frequent and sparsely represented keywords.
arXiv Detail & Related papers (2022-09-28T11:31:43Z) - HETFORMER: Heterogeneous Transformer with Sparse Attention for Long-Text
Extractive Summarization [57.798070356553936]
HETFORMER is a Transformer-based pre-trained model with multi-granularity sparse attentions for extractive summarization.
Experiments on both single- and multi-document summarization tasks show that HETFORMER achieves state-of-the-art performance in Rouge F1.
arXiv Detail & Related papers (2021-10-12T22:42:31Z) - CitationIE: Leveraging the Citation Graph for Scientific Information
Extraction [89.33938657493765]
We use the citation graph of referential links between citing and cited papers.
We observe a sizable improvement in end-to-end information extraction over the state-of-the-art.
arXiv Detail & Related papers (2021-06-03T03:00:12Z) - Enhancing Scientific Papers Summarization with Citation Graph [78.65955304229863]
We redefine the task of scientific papers summarization by utilizing their citation graph.
We construct a novel scientific papers summarization dataset Semantic Scholar Network (SSN) which contains 141K research papers in different domains.
Our model can achieve competitive performance when compared with the pretrained models.
arXiv Detail & Related papers (2021-04-07T11:13:35Z) - Topic-Centric Unsupervised Multi-Document Summarization of Scientific
and News Articles [3.0504782036247438]
We propose a topic-centric unsupervised multi-document summarization framework to generate abstractive summaries.
The proposed algorithm generates an abstractive summary by developing salient language unit selection and text generation techniques.
Our approach matches the state-of-the-art when evaluated on automated extractive evaluation metrics and performs better for abstractive summarization on five human evaluation metrics.
arXiv Detail & Related papers (2020-11-03T04:04:21Z) - Enhancing Extractive Text Summarization with Topic-Aware Graph Neural
Networks [21.379555672973975]
This paper proposes a graph neural network (GNN)-based extractive summarization model.
Our model integrates a joint neural topic model (NTM) to discover latent topics, which can provide document-level features for sentence selection.
The experimental results demonstrate that our model achieves substantially state-of-the-art results on CNN/DM and NYT datasets.
arXiv Detail & Related papers (2020-10-13T09:30:04Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.