Improving Topic Segmentation by Injecting Discourse Dependencies

• URL: http://arxiv.org/abs/2209.08626v1
• Date: Sun, 18 Sep 2022 18:22:25 GMT
• Title: Improving Topic Segmentation by Injecting Discourse Dependencies
• Authors: Linzi Xing, Patrick Huber, Giuseppe Carenini
• Abstract summary: We present a discourse-aware neural topic segmentation model with the injection of above-sentence discourse dependency structures. Our empirical study on English evaluation datasets shows that injecting above-sentence discourse structures to a neural topic segmenter can substantially improve its performances.
• Score: 29.353285741379334
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent neural supervised topic segmentation models achieve distinguished superior effectiveness over unsupervised methods, with the availability of large-scale training corpora sampled from Wikipedia. These models may, however, suffer from limited robustness and transferability caused by exploiting simple linguistic cues for prediction, but overlooking more important inter-sentential topical consistency. To address this issue, we present a discourse-aware neural topic segmentation model with the injection of above-sentence discourse dependency structures to encourage the model make topic boundary prediction based more on the topical consistency between sentences. Our empirical study on English evaluation datasets shows that injecting above-sentence discourse structures to a neural topic segmenter with our proposed strategy can substantially improve its performances on intra-domain and out-of-domain data, with little increase of model's complexity.

Related papers

• How Well Do Text Embedding Models Understand Syntax? [50.440590035493074]
  The ability of text embedding models to generalize across a wide range of syntactic contexts remains under-explored. Our findings reveal that existing text embedding models have not sufficiently addressed these syntactic understanding challenges. We propose strategies to augment the generalization ability of text embedding models in diverse syntactic scenarios.
  arXiv  Detail & Related papers  (2023-11-14T08:51:00Z)
• Coherent Entity Disambiguation via Modeling Topic and Categorical Dependency [87.16283281290053]
  Previous entity disambiguation (ED) methods adopt a discriminative paradigm, where prediction is made based on matching scores between mention context and candidate entities. We propose CoherentED, an ED system equipped with novel designs aimed at enhancing the coherence of entity predictions. We achieve new state-of-the-art results on popular ED benchmarks, with an average improvement of 1.3 F1 points.
  arXiv  Detail & Related papers  (2023-11-06T16:40:13Z)
• Diversity-Aware Coherence Loss for Improving Neural Topic Models [20.98172300869239]
  We propose a novel diversity-aware coherence loss that encourages the model to learn corpus-level coherence scores. Experimental results on multiple datasets show that our method significantly improves the performance of neural topic models.
  arXiv  Detail & Related papers  (2023-05-25T16:01:56Z)
• Topic-driven Distant Supervision Framework for Macro-level Discourse Parsing [72.14449502499535]
  The task of analyzing the internal rhetorical structure of texts is a challenging problem in natural language processing. Despite the recent advances in neural models, the lack of large-scale, high-quality corpora for training remains a major obstacle. Recent studies have attempted to overcome this limitation by using distant supervision.
  arXiv  Detail & Related papers  (2023-05-23T07:13:51Z)
• Constructing Word-Context-Coupled Space Aligned with Associative Knowledge Relations for Interpretable Language Modeling [0.0]
  The black-box structure of the deep neural network in pre-trained language models seriously limits the interpretability of the language modeling process. A Word-Context-Coupled Space (W2CSpace) is proposed by introducing the alignment processing between uninterpretable neural representation and interpretable statistical logic. Our language model can achieve better performance and highly credible interpretable ability compared to related state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-05-19T09:26:02Z)
• Towards Domain-Independent Supervised Discourse Parsing Through Gradient Boosting [30.615883375573432]
  We present a new, supervised paradigm directly tackling the domain adaptation issue in discourse parsing. Specifically, we introduce the first fully supervised discourse framework designed to alleviate the domain dependency through a staged model of weak gradient classifiers.
  arXiv  Detail & Related papers  (2022-10-18T03:44:27Z)
• Learning Neural Causal Models with Active Interventions [83.44636110899742]
  We introduce an active intervention-targeting mechanism which enables a quick identification of the underlying causal structure of the data-generating process. Our method significantly reduces the required number of interactions compared with random intervention targeting. We demonstrate superior performance on multiple benchmarks from simulated to real-world data.
  arXiv  Detail & Related papers  (2021-09-06T13:10:37Z)
• Improving Unsupervised Dialogue Topic Segmentation with Utterance-Pair Coherence Scoring [8.31009800792799]
  We present a strategy to generate a training corpus for utterance-pair coherence scoring. Then, we train a BERT-based neural utterance-pair coherence model with the obtained training corpus. Finally, such model is used to measure the topical relevance between utterances, acting as the basis of the segmentation inference.
  arXiv  Detail & Related papers  (2021-06-12T08:49:20Z)
• High-order Semantic Role Labeling [86.29371274587146]
  This paper introduces a high-order graph structure for the neural semantic role labeling model. It enables the model to explicitly consider not only the isolated predicate-argument pairs but also the interaction between the predicate-argument pairs. Experimental results on 7 languages of the CoNLL-2009 benchmark show that the high-order structural learning techniques are beneficial to the strong performing SRL models.
  arXiv  Detail & Related papers  (2020-10-09T15:33:54Z)
• Improving Context Modeling in Neural Topic Segmentation [18.92944038749279]
  We enhance a segmenter based on a hierarchical attention BiLSTM network to better model context. Our optimized segmenter outperforms SOTA approaches when trained and tested on three datasets.
  arXiv  Detail & Related papers  (2020-10-07T03:40:49Z)
• Neural Data-to-Text Generation via Jointly Learning the Segmentation and Correspondence [48.765579605145454]
  We propose to explicitly segment target text into fragment units and align them with their data correspondences. The resulting architecture maintains the same expressive power as neural attention models. On both E2E and WebNLG benchmarks, we show the proposed model consistently outperforms its neural attention counterparts.
  arXiv  Detail & Related papers  (2020-05-03T14:28:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.