RGAT: A Deeper Look into Syntactic Dependency Information for Coreference Resolution

• URL: http://arxiv.org/abs/2309.04977v1
• Date: Sun, 10 Sep 2023 09:46:38 GMT
• Title: RGAT: A Deeper Look into Syntactic Dependency Information for Coreference Resolution
• Authors: Yuan Meng, Xuhao Pan, Jun Chang and Yue Wang
• Abstract summary: We propose an end-to-end resolution that combines pre-trained BERT with a Syntactic Relation Graph Attention Network (RGAT) In particular, the RGAT model is first proposed, then used to understand the syntactic dependency graph and learn better task-specific syntactic embeddings. An integrated architecture incorporating BERT embeddings and syntactic embeddings is constructed to generate blending representations for the downstream task.
• Score: 8.017036537163008
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Although syntactic information is beneficial for many NLP tasks, combining it with contextual information between words to solve the coreference resolution problem needs to be further explored. In this paper, we propose an end-to-end parser that combines pre-trained BERT with a Syntactic Relation Graph Attention Network (RGAT) to take a deeper look into the role of syntactic dependency information for the coreference resolution task. In particular, the RGAT model is first proposed, then used to understand the syntactic dependency graph and learn better task-specific syntactic embeddings. An integrated architecture incorporating BERT embeddings and syntactic embeddings is constructed to generate blending representations for the downstream task. Our experiments on a public Gendered Ambiguous Pronouns (GAP) dataset show that with the supervision learning of the syntactic dependency graph and without fine-tuning the entire BERT, we increased the F1-score of the previous best model (RGCN-with-BERT) from 80.3% to 82.5%, compared to the F1-score by single BERT embeddings from 78.5% to 82.5%. Experimental results on another public dataset - OntoNotes 5.0 demonstrate that the performance of the model is also improved by incorporating syntactic dependency information learned from RGAT.

Related papers

• Syntactic Knowledge via Graph Attention with BERT in Machine Translation [0.0]
  We propose Syntactic knowledge via Graph attention with BERT (SGB) in Machine Translation (MT) scenarios. Our experiments use gold syntax-annotation sentences and Quality Estimation (QE) model to obtain interpretability of translation quality improvement. Experiments show that the proposed SGB engines improve translation quality across the three MT tasks without sacrificing BLEU scores.
  arXiv  Detail & Related papers  (2023-05-22T18:56:14Z)
• Sentence-Level Relation Extraction via Contrastive Learning with Descriptive Relation Prompts [1.5736899098702974]
  We propose a new paradigm, Contrastive Learning with Descriptive Relation Prompts(CTL-), to jointly consider entity information, relational knowledge and entity type restrictions. The CTL- obtains a competitive F1-score of 76.7% on TACRED. The new presented paradigm achieves F1-scores of 85.8% and 91.6% on TACREV and Re-TACRED respectively, which are both the state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-04-11T02:15:13Z)
• Enriching Relation Extraction with OpenIE [70.52564277675056]
  Relation extraction (RE) is a sub-discipline of information extraction (IE) In this work, we explore how recent approaches for open information extraction (OpenIE) may help to improve the task of RE. Our experiments over two annotated corpora, KnowledgeNet and FewRel, demonstrate the improved accuracy of our enriched models.
  arXiv  Detail & Related papers  (2022-12-19T11:26:23Z)
• SAIS: Supervising and Augmenting Intermediate Steps for Document-Level Relation Extraction [51.27558374091491]
  We propose to explicitly teach the model to capture relevant contexts and entity types by supervising and augmenting intermediate steps (SAIS) for relation extraction. Based on a broad spectrum of carefully designed tasks, our proposed SAIS method not only extracts relations of better quality due to more effective supervision, but also retrieves the corresponding supporting evidence more accurately.
  arXiv  Detail & Related papers  (2021-09-24T17:37:35Z)
• Exploiting Global Contextual Information for Document-level Named Entity Recognition [46.99922251839363]
  We propose a model called Global Context enhanced Document-level NER (GCDoc) At word-level, a document graph is constructed to model a wider range of dependencies between words. At sentence-level, for appropriately modeling wider context beyond single sentence, we employ a cross-sentence module. Our model reaches F1 score of 92.22 (93.40 with BERT) on CoNLL 2003 dataset and 88.32 (90.49 with BERT) on Ontonotes 5.0 dataset.
  arXiv  Detail & Related papers  (2021-06-02T01:52:07Z)
• A Frustratingly Easy Approach for Entity and Relation Extraction [25.797992240847833]
  We present a simple pipelined approach for entity and relation extraction. We establish the new state-of-the-art on standard benchmarks (ACE04, ACE05 and SciERC) Our approach essentially builds on two independent encoders and merely uses the entity model to construct the input for the relation model.
  arXiv  Detail & Related papers  (2020-10-24T07:14:01Z)
• Syntactic Structure Distillation Pretraining For Bidirectional Encoders [49.483357228441434]
  We introduce a knowledge distillation strategy for injecting syntactic biases into BERT pretraining. We distill the approximate marginal distribution over words in context from the syntactic LM. Our findings demonstrate the benefits of syntactic biases, even in representation learners that exploit large amounts of data.
  arXiv  Detail & Related papers  (2020-05-27T16:44:01Z)
• Coreferential Reasoning Learning for Language Representation [88.14248323659267]
  We present CorefBERT, a novel language representation model that can capture the coreferential relations in context. The experimental results show that, compared with existing baseline models, CorefBERT can achieve significant improvements consistently on various downstream NLP tasks.
  arXiv  Detail & Related papers  (2020-04-15T03:57:45Z)
• AMR Parsing via Graph-Sequence Iterative Inference [62.85003739964878]
  We propose a new end-to-end model that treats AMR parsing as a series of dual decisions on the input sequence and the incrementally constructed graph. We show that the answers to these two questions are mutually causalities. We design a model based on iterative inference that helps achieve better answers in both perspectives, leading to greatly improved parsing accuracy.
  arXiv  Detail & Related papers  (2020-04-12T09:15:21Z)
• A Dependency Syntactic Knowledge Augmented Interactive Architecture for End-to-End Aspect-based Sentiment Analysis [73.74885246830611]
  We propose a novel dependency syntactic knowledge augmented interactive architecture with multi-task learning for end-to-end ABSA. This model is capable of fully exploiting the syntactic knowledge (dependency relations and types) by leveraging a well-designed Dependency Relation Embedded Graph Convolutional Network (DreGcn) Extensive experimental results on three benchmark datasets demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2020-04-04T14:59:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.