Fine-Grained Evaluation for Implicit Discourse Relation Recognition

• URL: http://arxiv.org/abs/2503.05326v1
• Date: Fri, 07 Mar 2025 11:10:33 GMT
• Title: Fine-Grained Evaluation for Implicit Discourse Relation Recognition
• Authors: Xinyi Cai,
• Abstract summary: Implicit discourse relation recognition is a challenging task due to the absence of explicit discourse connectives between spans of text.<n>Recent pre-trained language models have achieved great success on this task.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Implicit discourse relation recognition is a challenging task in discourse analysis due to the absence of explicit discourse connectives between spans of text. Recent pre-trained language models have achieved great success on this task. However, there is no fine-grained analysis of the performance of these pre-trained language models for this task. Therefore, the difficulty and possible directions of this task is unclear. In this paper, we deeply analyze the model prediction, attempting to find out the difficulty for the pre-trained language models and the possible directions of this task. In addition to having an in-depth analysis for this task by using pre-trained language models, we semi-manually annotate data to add relatively high-quality data for the relations with few annotated examples in PDTB 3.0. The annotated data significantly help improve implicit discourse relation recognition for level-2 senses.

Related papers

• ImpScore: A Learnable Metric For Quantifying The Implicitness Level of Sentence [40.4052848203136]
  Implicit language is essential for natural language processing systems to achieve precise text understanding and facilitate natural interactions with users.<n>This paper develops a scalar metric that quantifies the implicitness level of language without relying on external references.<n>We validate ImpScore through a user study that compares its assessments with human evaluations on out-of-distribution data.
  arXiv  Detail & Related papers  (2024-11-07T20:23:29Z)
• Making Retrieval-Augmented Language Models Robust to Irrelevant Context [55.564789967211844]
  An important desideratum of RALMs, is that retrieved information helps model performance when it is relevant. Recent work has shown that retrieval augmentation can sometimes have a negative effect on performance.
  arXiv  Detail & Related papers  (2023-10-02T18:52:35Z)
• On Robustness of Prompt-based Semantic Parsing with Large Pre-trained Language Model: An Empirical Study on Codex [48.588772371355816]
  This paper presents the first empirical study on the adversarial robustness of a large prompt-based language model of code, codex. Our results demonstrate that the state-of-the-art (SOTA) code-language models are vulnerable to carefully crafted adversarial examples.
  arXiv  Detail & Related papers  (2023-01-30T13:21:00Z)
• Pre-trained Sentence Embeddings for Implicit Discourse Relation Classification [26.973476248983477]
  Implicit discourse relations bind smaller linguistic units into coherent texts. We explore the utility of pre-trained sentence embeddings as base representations in a neural network for implicit discourse relation sense classification.
  arXiv  Detail & Related papers  (2022-10-20T04:17:03Z)
• A Latent-Variable Model for Intrinsic Probing [93.62808331764072]
  We propose a novel latent-variable formulation for constructing intrinsic probes. We find empirical evidence that pre-trained representations develop a cross-lingually entangled notion of morphosyntax.
  arXiv  Detail & Related papers  (2022-01-20T15:01:12Z)
• Leveraging Pre-trained Language Model for Speech Sentiment Analysis [58.78839114092951]
  We explore the use of pre-trained language models to learn sentiment information of written texts for speech sentiment analysis. We propose a pseudo label-based semi-supervised training strategy using a language model on an end-to-end speech sentiment approach.
  arXiv  Detail & Related papers  (2021-06-11T20:15:21Z)
• Let's be explicit about that: Distant supervision for implicit discourse relation classification via connective prediction [0.0]
  In implicit discourse relation classification, we want to predict the relation between adjacent sentences in the absence of any overt discourse connectives. We sidestep the lack of data through explicitation of implicit relations to reduce the task to two sub-problems: language modeling and explicit discourse relation classification. Our experimental results show that this method can even marginally outperform the state-of-the-art, in spite of being much simpler than alternative models of comparable performance.
  arXiv  Detail & Related papers  (2021-06-06T17:57:32Z)
• ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learning [97.10875695679499]
  We propose a novel contrastive learning framework named ERICA in pre-training phase to obtain a deeper understanding of the entities and their relations in text. Experimental results demonstrate that our proposed ERICA framework achieves consistent improvements on several document-level language understanding tasks.
  arXiv  Detail & Related papers  (2020-12-30T03:35:22Z)
• Analysis and Evaluation of Language Models for Word Sense Disambiguation [18.001457030065712]
  Transformer-based language models have taken many fields in NLP by storm. BERT can accurately capture high-level sense distinctions, even when a limited number of examples is available for each word sense. BERT and its derivatives dominate most of the existing evaluation benchmarks.
  arXiv  Detail & Related papers  (2020-08-26T15:07:07Z)
• Labeling Explicit Discourse Relations using Pre-trained Language Models [0.0]
  State-of-the-art models achieve slightly above 45% of F-score by using hand-crafted features. We find that the pre-trained language models, when finetuned, are powerful enough to replace the linguistic features. This is the first time when a model outperforms the knowledge intensive models without employing any linguistic features.
  arXiv  Detail & Related papers  (2020-06-21T17:18:01Z)
• Data Augmentation for Spoken Language Understanding via Pretrained Language Models [113.56329266325902]
  Training of spoken language understanding (SLU) models often faces the problem of data scarcity. We put forward a data augmentation method using pretrained language models to boost the variability and accuracy of generated utterances.
  arXiv  Detail & Related papers  (2020-04-29T04:07:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.