DocSum: Domain-Adaptive Pre-training for Document Abstractive Summarization

• URL: http://arxiv.org/abs/2412.08196v1
• Date: Wed, 11 Dec 2024 08:36:50 GMT
• Title: DocSum: Domain-Adaptive Pre-training for Document Abstractive Summarization
• Authors: Phan Phuong Mai Chau, Souhail Bakkali, Antoine Doucet,
• Abstract summary: Abstractive summarization has made significant strides in condensing and rephrasing large volumes of text into coherent summaries. Existing models often struggle to adapt to the intricate structure and specialized content of such documents. We introduce DocSum, a domain-adaptive abstractive summarization framework tailored for administrative documents.
• Score: 2.8201999897313015
• License:
• Abstract: Abstractive summarization has made significant strides in condensing and rephrasing large volumes of text into coherent summaries. However, summarizing administrative documents presents unique challenges due to domain-specific terminology, OCR-generated errors, and the scarcity of annotated datasets for model fine-tuning. Existing models often struggle to adapt to the intricate structure and specialized content of such documents. To address these limitations, we introduce DocSum, a domain-adaptive abstractive summarization framework tailored for administrative documents. Leveraging pre-training on OCR-transcribed text and fine-tuning with an innovative integration of question-answer pairs, DocSum enhances summary accuracy and relevance. This approach tackles the complexities inherent in administrative content, ensuring outputs that align with real-world business needs. To evaluate its capabilities, we define a novel downstream task setting-Document Abstractive Summarization-which reflects the practical requirements of business and organizational settings. Comprehensive experiments demonstrate DocSum's effectiveness in producing high-quality summaries, showcasing its potential to improve decision-making and operational workflows across the public and private sectors.

Related papers

• A Rhetorical Relations-Based Framework for Tailored Multimedia Document Summarization [0.0]
  This paper introduces a novel framework for multimedia document summarization. The framework capitalizes on the inherent structure of the document to craft coherent and succinct summaries. Weighting algorithms are employed to assign significance values to document units, thereby enabling effective ranking and selection of relevant content.
  arXiv  Detail & Related papers  (2024-12-26T09:29:59Z)
• Beyond Relevant Documents: A Knowledge-Intensive Approach for Query-Focused Summarization using Large Language Models [27.90653125902507]
  We propose a knowledge-intensive approach that reframes query-focused summarization as a knowledge-intensive task setup. The retrieval module efficiently retrieves potentially relevant documents from a large-scale knowledge corpus. The summarization controller seamlessly integrates a powerful large language model (LLM)-based summarizer with a carefully tailored prompt.
  arXiv  Detail & Related papers  (2024-08-19T18:54:20Z)
• Write Summary Step-by-Step: A Pilot Study of Stepwise Summarization [48.57273563299046]
  We propose the task of Stepwise Summarization, which aims to generate a new appended summary each time a new document is proposed. The appended summary should not only summarize the newly added content but also be coherent with the previous summary. We show that SSG achieves state-of-the-art performance in terms of both automatic metrics and human evaluations.
  arXiv  Detail & Related papers  (2024-06-08T05:37:26Z)
• Interactive Editing for Text Summarization [30.46273082913698]
  REVISE is a framework designed to facilitate iterative editing and refinement of draft summaries by human writers. At its core, REVISE incorporates a modified fill-in-the-middle model with the encoder-decoder architecture.
  arXiv  Detail & Related papers  (2023-06-05T17:43:53Z)
• Document-Level Relation Extraction with Sentences Importance Estimation and Focusing [52.069206266557266]
  Document-level relation extraction (DocRE) aims to determine the relation between two entities from a document of multiple sentences. We propose a Sentence Estimation and Focusing (SIEF) framework for DocRE, where we design a sentence importance score and a sentence focusing loss. Experimental results on two domains show that our SIEF not only improves overall performance, but also makes DocRE models more robust.
  arXiv  Detail & Related papers  (2022-04-27T03:20:07Z)
• Controllable Summarization with Constrained Markov Decision Process [50.04321779376415]
  We study controllable text summarization which allows users to gain control on a particular attribute. We propose a novel training framework based on Constrained Markov Decision Process (CMDP) Our framework can be applied to control important attributes of summarization, including length, covered entities, and abstractiveness.
  arXiv  Detail & Related papers  (2021-08-07T09:12:53Z)
• Entity-level Factual Consistency of Abstractive Text Summarization [26.19686599842915]
  Key challenge for abstractive summarization is ensuring factual consistency of the generated summary with respect to the original document. We propose a set of new metrics to quantify the entity-level factual consistency of generated summaries.
  arXiv  Detail & Related papers  (2021-02-18T03:07:28Z)
• Constrained Abstractive Summarization: Preserving Factual Consistency with Constrained Generation [93.87095877617968]
  We propose Constrained Abstractive Summarization (CAS), a general setup that preserves the factual consistency of abstractive summarization. We adopt lexically constrained decoding, a technique generally applicable to autoregressive generative models, to fulfill CAS. We observe up to 13.8 ROUGE-2 gains when only one manual constraint is used in interactive summarization.
  arXiv  Detail & Related papers  (2020-10-24T00:27:44Z)
• Summarizing Text on Any Aspects: A Knowledge-Informed Weakly-Supervised Approach [89.56158561087209]
  We study summarizing on arbitrary aspects relevant to the document. Due to the lack of supervision data, we develop a new weak supervision construction method and an aspect modeling scheme. Experiments show our approach achieves performance boosts on summarizing both real and synthetic documents.
  arXiv  Detail & Related papers  (2020-10-14T03:20:46Z)
• Enhancing Factual Consistency of Abstractive Summarization [57.67609672082137]
  We propose a fact-aware summarization model FASum to extract and integrate factual relations into the summary generation process. We then design a factual corrector model FC to automatically correct factual errors from summaries generated by existing systems.
  arXiv  Detail & Related papers  (2020-03-19T07:36:10Z)
• StructSum: Summarization via Structured Representations [27.890477913486787]
  Abstractive text summarization aims at compressing the information of a long source document into a condensed summary. Despite advances in modeling techniques, abstractive summarization models still suffer from several key challenges. We propose a framework based on document-level structure induction for summarization to address these challenges.
  arXiv  Detail & Related papers  (2020-03-01T20:32:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.