Incorporating Distributions of Discourse Structure for Long Document Abstractive Summarization

• URL: http://arxiv.org/abs/2305.16784v1
• Date: Fri, 26 May 2023 09:51:47 GMT
• Title: Incorporating Distributions of Discourse Structure for Long Document Abstractive Summarization
• Authors: Dongqi Pu, Yifan Wang, Vera Demberg
• Abstract summary: This paper introduces the 'RSTformer', a novel summarization model that comprehensively incorporates both the types and uncertainty of rhetorical relations. Our RST-attention mechanism, rooted in document-level rhetorical structure, is an extension of the recently devised Longformer framework.
• Score: 11.168330694255404
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: For text summarization, the role of discourse structure is pivotal in discerning the core content of a text. Regrettably, prior studies on incorporating Rhetorical Structure Theory (RST) into transformer-based summarization models only consider the nuclearity annotation, thereby overlooking the variety of discourse relation types. This paper introduces the 'RSTformer', a novel summarization model that comprehensively incorporates both the types and uncertainty of rhetorical relations. Our RST-attention mechanism, rooted in document-level rhetorical structure, is an extension of the recently devised Longformer framework. Through rigorous evaluation, the model proposed herein exhibits significant superiority over state-of-the-art models, as evidenced by its notable performance on several automatic metrics and human evaluation.

Related papers

• RST-LoRA: A Discourse-Aware Low-Rank Adaptation for Long Document Abstractive Summarization [19.214041945441434]
  This paper introduces RST-LoRA and proposes four RST-aware variants to explicitly incorporate RST into the LoRA model. Our empirical evaluation demonstrates that incorporating the type and uncertainty of rhetorical relations can complementarily enhance the performance of LoRA.
  arXiv  Detail & Related papers  (2024-05-01T17:37:50Z)
• How Well Do Text Embedding Models Understand Syntax? [50.440590035493074]
  The ability of text embedding models to generalize across a wide range of syntactic contexts remains under-explored. Our findings reveal that existing text embedding models have not sufficiently addressed these syntactic understanding challenges. We propose strategies to augment the generalization ability of text embedding models in diverse syntactic scenarios.
  arXiv  Detail & Related papers  (2023-11-14T08:51:00Z)
• Controllable Topic-Focused Abstractive Summarization [57.8015120583044]
  Controlled abstractive summarization focuses on producing condensed versions of a source article to cover specific aspects. This paper presents a new Transformer-based architecture capable of producing topic-focused summaries.
  arXiv  Detail & Related papers  (2023-11-12T03:51:38Z)
• Model Criticism for Long-Form Text Generation [113.13900836015122]
  We apply a statistical tool, model criticism in latent space, to evaluate the high-level structure of generated text. We perform experiments on three representative aspects of high-level discourse -- coherence, coreference, and topicality. We find that transformer-based language models are able to capture topical structures but have a harder time maintaining structural coherence or modeling coreference.
  arXiv  Detail & Related papers  (2022-10-16T04:35:58Z)
• Knowledge-Aware Bayesian Deep Topic Model [50.58975785318575]
  We propose a Bayesian generative model for incorporating prior domain knowledge into hierarchical topic modeling. Our proposed model efficiently integrates the prior knowledge and improves both hierarchical topic discovery and document representation.
  arXiv  Detail & Related papers  (2022-09-20T09:16:05Z)
• RSTGen: Imbuing Fine-Grained Interpretable Control into Long-FormText Generators [26.27412809287025]
  RSTGen is a framework that controls the discourse structure, semantics and topics of generated text. We demonstrate our model's ability to control structural discourse and semantic features of generated text in open generation evaluation.
  arXiv  Detail & Related papers  (2022-05-25T09:06:04Z)
• Revise and Resubmit: An Intertextual Model of Text-based Collaboration in Peer Review [52.359007622096684]
  Peer review is a key component of the publishing process in most fields of science. Existing NLP studies focus on the analysis of individual texts. editorial assistance often requires modeling interactions between pairs of texts.
  arXiv  Detail & Related papers  (2022-04-22T16:39:38Z)
• Dialogue Summarization with Supporting Utterance Flow Modeling and Fact Regularization [58.965859508695225]
  We propose an end-to-end neural model for dialogue summarization with two novel modules. The supporting utterance flow modeling helps to generate a coherent summary by smoothly shifting the focus from the former utterances to the later ones. The fact regularization encourages the generated summary to be factually consistent with the ground-truth summary during model training.
  arXiv  Detail & Related papers  (2021-08-03T03:09:25Z)
• Semiparametric Latent Topic Modeling on Consumer-Generated Corpora [0.0]
  This paper proposes semiparametric topic model, a two-step approach utilizing nonnegative matrix factorization and semiparametric regression in topic modeling. The model enables the reconstruction of sparse topic structures in the corpus and provides a generative model for predicting topics in new documents entering the corpus. In an actual consumer feedback corpus, the model also demonstrably provides interpretable and useful topic definitions comparable with those produced by other methods.
  arXiv  Detail & Related papers  (2021-07-13T00:22:02Z)
• Decomposing and Recomposing Event Structure [4.270553193574436]
  We induce this jointly with role, entity type and event-level semantic graphs. We identify sets of types that align closely with previous theoretically-motivated document-level generative models.
  arXiv  Detail & Related papers  (2021-03-18T17:16:43Z)
• StructSum: Summarization via Structured Representations [27.890477913486787]
  Abstractive text summarization aims at compressing the information of a long source document into a condensed summary. Despite advances in modeling techniques, abstractive summarization models still suffer from several key challenges. We propose a framework based on document-level structure induction for summarization to address these challenges.
  arXiv  Detail & Related papers  (2020-03-01T20:32:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.