SKT5SciSumm -- Revisiting Extractive-Generative Approach for Multi-Document Scientific Summarization

• URL: http://arxiv.org/abs/2402.17311v2
• Date: Fri, 27 Sep 2024 05:38:03 GMT
• Title: SKT5SciSumm -- Revisiting Extractive-Generative Approach for Multi-Document Scientific Summarization
• Authors: Huy Quoc To, Ming Liu, Guangyan Huang, Hung-Nghiep Tran, Andr'e Greiner-Petter, Felix Beierle, Akiko Aizawa,
• Abstract summary: We propose SKT5SciSumm - a hybrid framework for multi-document scientific summarization (MDSS) We leverage the Sentence-Transformer version of Scientific Paper Embeddings using Citation-Informed Transformers (SPECTER) to encode and represent textual sentences. We employ the T5 family of models to generate abstractive summaries using extracted sentences.
• Score: 24.051692189473723
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Summarization for scientific text has shown significant benefits both for the research community and human society. Given the fact that the nature of scientific text is distinctive and the input of the multi-document summarization task is substantially long, the task requires sufficient embedding generation and text truncation without losing important information. To tackle these issues, in this paper, we propose SKT5SciSumm - a hybrid framework for multi-document scientific summarization (MDSS). We leverage the Sentence-Transformer version of Scientific Paper Embeddings using Citation-Informed Transformers (SPECTER) to encode and represent textual sentences, allowing for efficient extractive summarization using k-means clustering. We employ the T5 family of models to generate abstractive summaries using extracted sentences. SKT5SciSumm achieves state-of-the-art performance on the Multi-XScience dataset. Through extensive experiments and evaluation, we showcase the benefits of our model by using less complicated models to achieve remarkable results, thereby highlighting its potential in advancing the field of multi-document summarization for scientific text.

Related papers

• Synthesizing Scientific Summaries: An Extractive and Abstractive Approach [0.5904095466127044]
  We propose a hybrid methodology for research paper summarisation. We use two models based on unsupervised learning for the extraction stage and two transformer language models. We find that using certain combinations of hyper parameters, it is possible for automated summarisation systems to exceed the abstractiveness of summaries written by humans.
  arXiv  Detail & Related papers  (2024-07-29T08:21:42Z)
• Towards Enhancing Coherence in Extractive Summarization: Dataset and Experiments with LLMs [70.15262704746378]
  We propose a systematically created human-annotated dataset consisting of coherent summaries for five publicly available datasets and natural language user feedback. Preliminary experiments with Falcon-40B and Llama-2-13B show significant performance improvements (10% Rouge-L) in terms of producing coherent summaries.
  arXiv  Detail & Related papers  (2024-07-05T20:25:04Z)
• MASSW: A New Dataset and Benchmark Tasks for AI-Assisted Scientific Workflows [58.56005277371235]
  We introduce MASSW, a comprehensive text dataset on Multi-Aspect Summarization of ScientificAspects. MASSW includes more than 152,000 peer-reviewed publications from 17 leading computer science conferences spanning the past 50 years. We demonstrate the utility of MASSW through multiple novel machine-learning tasks that can be benchmarked using this new dataset.
  arXiv  Detail & Related papers  (2024-06-10T15:19:09Z)
• Fusing Multimodal Signals on Hyper-complex Space for Extreme Abstractive Text Summarization (TL;DR) of Scientific Contents [26.32569293387399]
  We deal with a novel task of extreme abstractive text summarization (aka TL;DR generation) by leveraging multiple input modalities. The mTLDR dataset accompanies a total of 4,182 instances collected from various academic conference proceedings. We present mTLDRgen, an encoder-decoder-based model that employs a novel dual-fused hyper-complex Transformer.
  arXiv  Detail & Related papers  (2023-06-24T13:51:42Z)
• Keyword Extraction from Short Texts with~a~Text-To-Text Transfer Transformer [0.0]
  The paper explores the relevance of the Text-To-Text Transfer Transformer language model (T5) for Polish to the task of intrinsic and extrinsic keyword extraction from short text passages. We compare the results obtained by four different methods, i.e. plT5kw, extremeText, TermoPL, KeyBERT and conclude that the plT5kw model yields particularly promising results for both frequent and sparsely represented keywords.
  arXiv  Detail & Related papers  (2022-09-28T11:31:43Z)
• HETFORMER: Heterogeneous Transformer with Sparse Attention for Long-Text Extractive Summarization [57.798070356553936]
  HETFORMER is a Transformer-based pre-trained model with multi-granularity sparse attentions for extractive summarization. Experiments on both single- and multi-document summarization tasks show that HETFORMER achieves state-of-the-art performance in Rouge F1.
  arXiv  Detail & Related papers  (2021-10-12T22:42:31Z)
• CitationIE: Leveraging the Citation Graph for Scientific Information Extraction [89.33938657493765]
  We use the citation graph of referential links between citing and cited papers. We observe a sizable improvement in end-to-end information extraction over the state-of-the-art.
  arXiv  Detail & Related papers  (2021-06-03T03:00:12Z)
• Enhancing Scientific Papers Summarization with Citation Graph [78.65955304229863]
  We redefine the task of scientific papers summarization by utilizing their citation graph. We construct a novel scientific papers summarization dataset Semantic Scholar Network (SSN) which contains 141K research papers in different domains. Our model can achieve competitive performance when compared with the pretrained models.
  arXiv  Detail & Related papers  (2021-04-07T11:13:35Z)
• Topic-Centric Unsupervised Multi-Document Summarization of Scientific and News Articles [3.0504782036247438]
  We propose a topic-centric unsupervised multi-document summarization framework to generate abstractive summaries. The proposed algorithm generates an abstractive summary by developing salient language unit selection and text generation techniques. Our approach matches the state-of-the-art when evaluated on automated extractive evaluation metrics and performs better for abstractive summarization on five human evaluation metrics.
  arXiv  Detail & Related papers  (2020-11-03T04:04:21Z)
• Enhancing Extractive Text Summarization with Topic-Aware Graph Neural Networks [21.379555672973975]
  This paper proposes a graph neural network (GNN)-based extractive summarization model. Our model integrates a joint neural topic model (NTM) to discover latent topics, which can provide document-level features for sentence selection. The experimental results demonstrate that our model achieves substantially state-of-the-art results on CNN/DM and NYT datasets.
  arXiv  Detail & Related papers  (2020-10-13T09:30:04Z)
• Extractive Summarization as Text Matching [123.09816729675838]
  This paper creates a paradigm shift with regard to the way we build neural extractive summarization systems. We formulate the extractive summarization task as a semantic text matching problem. We have driven the state-of-the-art extractive result on CNN/DailyMail to a new level (44.41 in ROUGE-1)
  arXiv  Detail & Related papers  (2020-04-19T08:27:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.