Less is More: Quality-Aware Training Data Selection for Scientific Summarization

• URL: http://arxiv.org/abs/2606.24828v1
• Date: Tue, 23 Jun 2026 17:12:06 GMT
• Title: Less is More: Quality-Aware Training Data Selection for Scientific Summarization
• Abstract summary: We release one of the largest biomedical and life science datasets for long-document summarization.<n>We show that author-written abstracts vary in their alignment with the full article and that these quality signals can guide training-data selection.<n>Our findings suggest that reference quality is an important factor in scientific summarization and that quality-aware data selection can improve training efficiency.
• Score: 4.893896929103367
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Scientific long-document summarization datasets commonly treat author-written abstracts as gold reference summaries, although their quality and alignment with the source article vary. At the same time, publicly available scientific summarization datasets remain limited in scale and structure for modern long-context models. In this work, we address both challenges by a) constructing and releasing one of the largest biomedical and life science datasets for long-document summarization, containing 1.88 million PMC articles, and b) analyzing the reference quality of author-written abstracts with source-grounded and model-based metrics. We show that author-written abstracts vary in their alignment with the full article and that these quality signals can guide training-data selection. Training on selected high-quality subsets outperforms random sampling at matched training sizes and can match or exceed larger random subsets on factuality-oriented metrics. Our findings suggest that reference quality is an important factor in scientific summarization and that quality-aware data selection can improve training efficiency.

Related papers

• Fidelity-Diversity Metrics for Text [9.592420576478261]
  We build on work measuring the precision and recall of generative models to develop a pair of metrics.<n>We show that these metrics are able to disentangle a lack of fidelity from a lack of diversity in deficient candidate text.<n>In further experiments, our metrics detect diversity deficits in synthetic GSM8K-style math datasets.
  arXiv  Detail & Related papers  (2026-07-06T00:31:31Z)
• SciER: An Entity and Relation Extraction Dataset for Datasets, Methods, and Tasks in Scientific Documents [49.54155332262579]
  We release a new entity and relation extraction dataset for entities related to datasets, methods, and tasks in scientific articles. Our dataset contains 106 manually annotated full-text scientific publications with over 24k entities and 12k relations.
  arXiv  Detail & Related papers  (2024-10-28T15:56:49Z)
• Are Large Language Models Good Classifiers? A Study on Edit Intent Classification in Scientific Document Revisions [62.12545440385489]
  Large language models (LLMs) have brought substantial advancements in text generation, but their potential for enhancing classification tasks remains underexplored. We propose a framework for thoroughly investigating fine-tuning LLMs for classification, including both generation- and encoding-based approaches. We instantiate this framework in edit intent classification (EIC), a challenging and underexplored classification task.
  arXiv  Detail & Related papers  (2024-10-02T20:48:28Z)
• SurveySum: A Dataset for Summarizing Multiple Scientific Articles into a Survey Section [7.366861473623427]
  This paper introduces a novel dataset designed for summarizing multiple scientific articles into a section of a survey. Our contributions are: (1) SurveySum, a new dataset addressing the gap in domain-specific summarization tools; (2) two specific pipelines to summarize scientific articles into a section of a survey; and (3) the evaluation of these pipelines using multiple metrics to compare their performance.
  arXiv  Detail & Related papers  (2024-08-29T11:13:23Z)
• MMSci: A Dataset for Graduate-Level Multi-Discipline Multimodal Scientific Understanding [59.41495657570397]
  We present a comprehensive dataset compiled from Nature Communications articles covering 72 scientific fields.<n>We evaluated 19 proprietary and open-source models on two benchmark tasks, figure captioning and multiple-choice, and conducted human expert annotation.<n>Fine-tuning Qwen2-VL-7B with our task-specific data achieved better performance than GPT-4o and even human experts in multiple-choice evaluations.
  arXiv  Detail & Related papers  (2024-07-06T00:40:53Z)
• ACLSum: A New Dataset for Aspect-based Summarization of Scientific Publications [10.529898520273063]
  ACLSum is a novel summarization dataset carefully crafted and evaluated by domain experts. In contrast to previous datasets, ACLSum facilitates multi-aspect summarization of scientific papers.
  arXiv  Detail & Related papers  (2024-03-08T13:32:01Z)
• Exploring Precision and Recall to assess the quality and diversity of LLMs [82.21278402856079]
  We introduce a novel evaluation framework for Large Language Models (LLMs) such as textscLlama-2 and textscMistral. This approach allows for a nuanced assessment of the quality and diversity of generated text without the need for aligned corpora.
  arXiv  Detail & Related papers  (2024-02-16T13:53:26Z)
• QuRating: Selecting High-Quality Data for Training Language Models [64.83332850645074]
  We introduce QuRating, a method for selecting pre-training data that can capture human intuitions about data quality. In this paper, we investigate four qualities - writing style, required expertise, facts & trivia, and educational value. We train a Qur model to learn scalar ratings from pairwise judgments, and use it to annotate a 260B training corpus with quality ratings for each of the four criteria.
  arXiv  Detail & Related papers  (2024-02-15T06:36:07Z)
• A Novel Metric for Measuring Data Quality in Classification Applications (extended version) [0.0]
  We introduce and explain a novel metric to measure data quality. This metric is based on the correlated evolution between the classification performance and the deterioration of data. We provide an interpretation of each criterion and examples of assessment levels.
  arXiv  Detail & Related papers  (2023-12-13T11:20:09Z)
• Entity-driven Fact-aware Abstractive Summarization of Biomedical Literature [3.977582258550673]
  We propose an entity-driven fact-aware framework for training end-to-end transformer-based encoder-decoder models for abstractive summarization of biomedical articles. We conduct experiments using five state-of-the-art transformer-based models. The proposed approach is evaluated on ICD-11-Summ-1000, and PubMed-50k.
  arXiv  Detail & Related papers  (2022-03-30T00:34:56Z)
• Enhancing Scientific Papers Summarization with Citation Graph [78.65955304229863]
  We redefine the task of scientific papers summarization by utilizing their citation graph. We construct a novel scientific papers summarization dataset Semantic Scholar Network (SSN) which contains 141K research papers in different domains. Our model can achieve competitive performance when compared with the pretrained models.
  arXiv  Detail & Related papers  (2021-04-07T11:13:35Z)
• Unsupervised Opinion Summarization with Content Planning [58.5308638148329]
  We show that explicitly incorporating content planning in a summarization model yields output of higher quality. We also create synthetic datasets which are more natural, resembling real world document-summary pairs. Our approach outperforms competitive models in generating informative, coherent, and fluent summaries.
  arXiv  Detail & Related papers  (2020-12-14T18:41:58Z)
• Method and Dataset Entity Mining in Scientific Literature: A CNN + Bi-LSTM Model with Self-attention [21.93889297841459]
  We propose a novel entity recognition model, called MDER, which is able to effectively extract the method and dataset entities from scientific papers. We evaluate the proposed model on datasets constructed from the published papers of four research areas in computer science, i.e., NLP, CV, Data Mining and AI.
  arXiv  Detail & Related papers  (2020-10-26T13:38:43Z)
• Improving Zero and Few-Shot Abstractive Summarization with Intermediate Fine-tuning and Data Augmentation [101.26235068460551]
  Models pretrained with self-supervised objectives on large text corpora achieve state-of-the-art performance on English text summarization tasks. Models are typically fine-tuned on hundreds of thousands of data points, an infeasible requirement when applying summarization to new, niche domains. We introduce a novel and generalizable method, called WikiTransfer, for fine-tuning pretrained models for summarization in an unsupervised, dataset-specific manner.
  arXiv  Detail & Related papers  (2020-10-24T08:36:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.