Discontinuous Constituent Parsing with Pointer Networks

• URL: http://arxiv.org/abs/2002.01824v1
• Date: Wed, 5 Feb 2020 15:12:03 GMT
• Title: Discontinuous Constituent Parsing with Pointer Networks
• Authors: Daniel Fern\'andez-Gonz\'alez and Carlos G\'omez-Rodr\'iguez
• Abstract summary: discontinuous constituent trees are crucial for representing all grammatical phenomena of languages such as German. Recent advances in dependency parsing have shown that Pointer Networks excel in efficiently parsing syntactic relations between words in a sentence. We propose a novel neural network architecture that is able to generate the most accurate discontinuous constituent representations.
• Score: 0.34376560669160383
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: One of the most complex syntactic representations used in computational linguistics and NLP are discontinuous constituent trees, crucial for representing all grammatical phenomena of languages such as German. Recent advances in dependency parsing have shown that Pointer Networks excel in efficiently parsing syntactic relations between words in a sentence. This kind of sequence-to-sequence models achieve outstanding accuracies in building non-projective dependency trees, but its potential has not been proved yet on a more difficult task. We propose a novel neural network architecture that, by means of Pointer Networks, is able to generate the most accurate discontinuous constituent representations to date, even without the need of Part-of-Speech tagging information. To do so, we internally model discontinuous constituent structures as augmented non-projective dependency structures. The proposed approach achieves state-of-the-art results on the two widely-used NEGRA and TIGER benchmarks, outperforming previous work by a wide margin.

Related papers

• Entity-Aware Self-Attention and Contextualized GCN for Enhanced Relation Extraction in Long Sentences [5.453850739960517]
  We propose a novel model, Entity-aware Self-attention Contextualized GCN (ESC-GCN), which efficiently incorporates syntactic structure of input sentences and semantic context of sequences. Our model achieves encouraging performance as compared to existing dependency-based and sequence-based models.
  arXiv  Detail & Related papers  (2024-09-15T10:50:51Z)
• Semantic Loss Functions for Neuro-Symbolic Structured Prediction [74.18322585177832]
  We discuss the semantic loss, which injects knowledge about such structure, defined symbolically, into training. It is agnostic to the arrangement of the symbols, and depends only on the semantics expressed thereby. It can be combined with both discriminative and generative neural models.
  arXiv  Detail & Related papers  (2024-05-12T22:18:25Z)
• Incorporating Constituent Syntax for Coreference Resolution [50.71868417008133]
  We propose a graph-based method to incorporate constituent syntactic structures. We also explore to utilise higher-order neighbourhood information to encode rich structures in constituent trees. Experiments on the English and Chinese portions of OntoNotes 5.0 benchmark show that our proposed model either beats a strong baseline or achieves new state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-02-22T07:40:42Z)
• Discontinuous Grammar as a Foreign Language [0.7412445894287709]
  We extend the framework of sequence-to-sequence models for constituent parsing. We design several novelizations that can fully produce discontinuities. For the first time, we test a sequence-to-sequence model on the main discontinuous benchmarks.
  arXiv  Detail & Related papers  (2021-10-20T08:58:02Z)
• Infusing Finetuning with Semantic Dependencies [62.37697048781823]
  We show that, unlike syntax, semantics is not brought to the surface by today's pretrained models. We then use convolutional graph encoders to explicitly incorporate semantic parses into task-specific finetuning.
  arXiv  Detail & Related papers  (2020-12-10T01:27:24Z)
• Multitask Pointer Network for Multi-Representational Parsing [0.34376560669160383]
  We propose a transition-based approach that, by training a single model, can efficiently parse any input sentence with both constituent and dependency trees. We develop a Pointer Network architecture with two separate task-specific decoders and a common encoder, and follow a learning strategy to jointly train them.
  arXiv  Detail & Related papers  (2020-09-21T10:04:07Z)
• Transition-based Semantic Dependency Parsing with Pointer Networks [0.34376560669160383]
  We propose a transition system that can straightforwardly produce labelled directed acyclic graphs and perform semantic dependency parsing. We enhance our approach with deep contextualized word embeddings extracted from BERT. The resulting system not only outperforms all existing transition-based models, but also matches the best fully-supervised accuracy to date on SemEval 2015 18 English datasets.
  arXiv  Detail & Related papers  (2020-05-27T13:18:27Z)
• Exploiting Syntactic Structure for Better Language Modeling: A Syntactic Distance Approach [78.77265671634454]
  We make use of a multi-task objective, i.e., the models simultaneously predict words as well as ground truth parse trees in a form called "syntactic distances" Experimental results on the Penn Treebank and Chinese Treebank datasets show that when ground truth parse trees are provided as additional training signals, the model is able to achieve lower perplexity and induce trees with better quality.
  arXiv  Detail & Related papers  (2020-05-12T15:35:00Z)
• Representations of Syntax [MASK] Useful: Effects of Constituency and Dependency Structure in Recursive LSTMs [26.983602540576275]
  Sequence-based neural networks show significant sensitivity to syntactic structure, but they still perform less well on syntactic tasks than tree-based networks. We evaluate which of these two representational schemes more effectively introduces biases for syntactic structure. We show that a constituency-based network generalizes more robustly than a dependency-based one, and that combining the two types of structure does not yield further improvement.
  arXiv  Detail & Related papers  (2020-04-30T18:00:06Z)
• BURT: BERT-inspired Universal Representation from Twin Structure [89.82415322763475]
  BURT (BERT inspired Universal Representation from Twin Structure) is capable of generating universal, fixed-size representations for input sequences of any granularity. Our proposed BURT adopts the Siamese network, learning sentence-level representations from natural language inference dataset and word/phrase-level representations from paraphrasing dataset. We evaluate BURT across different granularities of text similarity tasks, including STS tasks, SemEval2013 Task 5(a) and some commonly used word similarity tasks.
  arXiv  Detail & Related papers  (2020-04-29T04:01:52Z)
• Coreferential Reasoning Learning for Language Representation [88.14248323659267]
  We present CorefBERT, a novel language representation model that can capture the coreferential relations in context. The experimental results show that, compared with existing baseline models, CorefBERT can achieve significant improvements consistently on various downstream NLP tasks.
  arXiv  Detail & Related papers  (2020-04-15T03:57:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.