Strongly Incremental Constituency Parsing with Graph Neural Networks
- URL: http://arxiv.org/abs/2010.14568v1
- Date: Tue, 27 Oct 2020 19:19:38 GMT
- Title: Strongly Incremental Constituency Parsing with Graph Neural Networks
- Authors: Kaiyu Yang, Jia Deng
- Abstract summary: Parsing sentences into syntax trees can benefit downstream applications in NLP.
Transition-baseds build trees by executing actions in a state transition system.
Existing transition-baseds are predominantly based on the shift-reduce transition system.
- Score: 70.16880251349093
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Parsing sentences into syntax trees can benefit downstream applications in
NLP. Transition-based parsers build trees by executing actions in a state
transition system. They are computationally efficient, and can leverage machine
learning to predict actions based on partial trees. However, existing
transition-based parsers are predominantly based on the shift-reduce transition
system, which does not align with how humans are known to parse sentences.
Psycholinguistic research suggests that human parsing is strongly incremental:
humans grow a single parse tree by adding exactly one token at each step. In
this paper, we propose a novel transition system called attach-juxtapose. It is
strongly incremental; it represents a partial sentence using a single tree;
each action adds exactly one token into the partial tree. Based on our
transition system, we develop a strongly incremental parser. At each step, it
encodes the partial tree using a graph neural network and predicts an action.
We evaluate our parser on Penn Treebank (PTB) and Chinese Treebank (CTB). On
PTB, it outperforms existing parsers trained with only constituency trees; and
it performs on par with state-of-the-art parsers that use dependency trees as
additional training data. On CTB, our parser establishes a new state of the
art. Code is available at
https://github.com/princeton-vl/attach-juxtapose-parser.
Related papers
- Integrating Supertag Features into Neural Discontinuous Constituent Parsing [0.0]
Traditional views of constituency demand that constituents consist of adjacent words, common in languages like German.
Transition-based parsing produces trees given raw text input using supervised learning on large annotated corpora.
arXiv Detail & Related papers (2024-10-11T12:28:26Z) - TreePiece: Faster Semantic Parsing via Tree Tokenization [2.1685554819849613]
TreePiece tokenizes a parse tree into subtrees and generates one subtree per decoding step.
On TopV2 benchmark, TreePiece shows 4.6 times faster decoding speed than standard AR.
arXiv Detail & Related papers (2023-03-30T05:44:44Z) - Characterizing Intrinsic Compositionality in Transformers with Tree
Projections [72.45375959893218]
neural models like transformers can route information arbitrarily between different parts of their input.
We show that transformers for three different tasks become more treelike over the course of training.
These trees are predictive of model behavior, with more tree-like models generalizing better on tests of compositional generalization.
arXiv Detail & Related papers (2022-11-02T17:10:07Z) - Unsupervised and Few-shot Parsing from Pretrained Language Models [56.33247845224995]
We propose an Unsupervised constituent Parsing model that calculates an Out Association score solely based on the self-attention weight matrix learned in a pretrained language model.
We extend the unsupervised models to few-shot parsing models that use a few annotated trees to learn better linear projection matrices for parsing.
Our few-shot parsing model FPIO trained with only 20 annotated trees outperforms a previous few-shot parsing method trained with 50 annotated trees.
arXiv Detail & Related papers (2022-06-10T10:29:15Z) - Penn-Helsinki Parsed Corpus of Early Modern English: First Parsing
Results and Analysis [2.8749014299466444]
We present the first parsing results on the Penn-Helsinki Parsed Corpus of Early Modern English (PPCEME), a 1.9 million word treebank.
We describe key features of PPCEME that make it challenging for parsing, including a larger and more varied set of function tags than in the Penn Treebank.
arXiv Detail & Related papers (2021-12-15T23:56:21Z) - SmBoP: Semi-autoregressive Bottom-up Semantic Parsing [44.802643057976354]
We propose a Semi-autoregressive Bottom-up (SmBoP) that constructs at decoding step $t$ the top-$K$ sub-trees of height $leq t$.
From an efficiency perspective, bottom-up parsing allows to decode all sub-trees of certain height in parallel, leading to logarithmic complexity runtime rather than linear.
We apply SmBoP on Spider, a challenging zero-shot semantic parsing benchmark, and show that SmBoP leads to a 2.2x speed-up in decoding time and a $$5x speed-up in training time
arXiv Detail & Related papers (2020-10-23T14:02:32Z) - Please Mind the Root: Decoding Arborescences for Dependency Parsing [67.71280539312536]
We analyze the output of state-of-the-arts on many languages from the Universal Dependency Treebank.
The worst constraint-violation rate we observe is 24%.
arXiv Detail & Related papers (2020-10-06T08:31:14Z) - Span-based Semantic Parsing for Compositional Generalization [53.24255235340056]
SpanBasedSP predicts a span tree over an input utterance, explicitly encoding how partial programs compose over spans in the input.
On GeoQuery, SCAN and CLOSURE, SpanBasedSP performs similarly to strong seq2seq baselines on random splits, but dramatically improves performance compared to baselines on splits that require compositional generalization.
arXiv Detail & Related papers (2020-09-13T16:42:18Z) - A Simple Global Neural Discourse Parser [61.728994693410954]
We propose a simple chart-based neural discourse that does not require any manually-crafted features and is based on learned span representations only.
We empirically demonstrate that our model achieves the best performance among globals, and comparable performance to state-of-art greedys.
arXiv Detail & Related papers (2020-09-02T19:28:40Z) - Parsing as Pretraining [13.03764728768944]
We first cast constituent and dependency parsing as sequence tagging.
We then use a single feed-forward layer to directly map word vectors to labels that encode a linearized tree.
arXiv Detail & Related papers (2020-02-05T08:43:02Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.