Neural Combinatory Constituency Parsing

• URL: http://arxiv.org/abs/2106.06689v1
• Date: Sat, 12 Jun 2021 05:14:16 GMT
• Title: Neural Combinatory Constituency Parsing
• Authors: Zhousi Chen, Longtu Zhang, Aizhan Imankulova, and Mamoru Komachi
• Abstract summary: Our models decompose the bottom-up parsing process into 1) classification of tags, labels, and binary orientations or chunks and 2) vector composition based on the computed orientations or chunks. The binary model achieves an F1 score of 92.54 on Penn Treebank, speeding at 1327.2 sents/sec. Both the models with XLNet provide near state-of-the-art accuracies for English.
• Score: 12.914521751805658
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose two fast neural combinatory models for constituency parsing: binary and multi-branching. Our models decompose the bottom-up parsing process into 1) classification of tags, labels, and binary orientations or chunks and 2) vector composition based on the computed orientations or chunks. These models have theoretical sub-quadratic complexity and empirical linear complexity. The binary model achieves an F1 score of 92.54 on Penn Treebank, speeding at 1327.2 sents/sec. Both the models with XLNet provide near state-of-the-art accuracies for English. Syntactic branching tendency and headedness of a language are observed during the training and inference processes for Penn Treebank, Chinese Treebank, and Keyaki Treebank (Japanese).

Related papers

• Universal Conceptual Structure in Neural Translation: Probing NLLB-200's Multilingual Geometry [0.0]
  We investigate the representation geometry of Meta's NLLB-200, a 200-language encoder-decoder Transformer.<n>We find that the model's embedding distances significantly correlate with phylogenetic distances from the Automated Similarity Judgment Program.<n>We release InterpretCognates, an open-source interactive toolkit for exploring these phenomena.
  arXiv  Detail & Related papers  (2026-02-27T22:51:01Z)
• High-order Joint Constituency and Dependency Parsing [15.697429723696011]
  We revisit the topic of jointly parsing constituency and dependency trees, i.e., to produce compatible constituency and dependency trees simultaneously for input sentences. We conduct experiments and analysis on seven languages, covering both rich-resource and low-resource scenarios.
  arXiv  Detail & Related papers  (2023-09-21T08:45:41Z)
• Hierarchical Phrase-based Sequence-to-Sequence Learning [94.10257313923478]
  We describe a neural transducer that maintains the flexibility of standard sequence-to-sequence (seq2seq) models while incorporating hierarchical phrases as a source of inductive bias during training and as explicit constraints during inference. Our approach trains two models: a discriminative derivation based on a bracketing grammar whose tree hierarchically aligns source and target phrases, and a neural seq2seq model that learns to translate the aligned phrases one-by-one.
  arXiv  Detail & Related papers  (2022-11-15T05:22:40Z)
• Characterizing Intrinsic Compositionality in Transformers with Tree Projections [72.45375959893218]
  neural models like transformers can route information arbitrarily between different parts of their input. We show that transformers for three different tasks become more treelike over the course of training. These trees are predictive of model behavior, with more tree-like models generalizing better on tests of compositional generalization.
  arXiv  Detail & Related papers  (2022-11-02T17:10:07Z)
• Towards Accurate Binary Neural Networks via Modeling Contextual Dependencies [52.691032025163175]
  Existing Binary Neural Networks (BNNs) operate mainly on local convolutions with binarization function. We present new designs of binary neural modules, which enables leading binary neural modules by a large margin.
  arXiv  Detail & Related papers  (2022-09-03T11:51:04Z)
• Learning from Non-Binary Constituency Trees via Tensor Decomposition [12.069862650316262]
  We introduce a new approach to deal with non-binary constituency trees. We show how a powerful composition function based on the canonical tensor decomposition can exploit such a rich structure. We experimentally assess its performance on different NLP tasks.
  arXiv  Detail & Related papers  (2020-11-02T10:06:59Z)
• Latent Tree Learning with Ordered Neurons: What Parses Does It Produce? [2.025491206574996]
  latent tree learning models can learn constituency parsing without exposure to human-annotated tree structures. ON-LSTM is trained on language modelling and has near-state-of-the-art performance on unsupervised parsing. We replicate the model with different restarts and examine their parses.
  arXiv  Detail & Related papers  (2020-10-10T07:12:48Z)
• Recursive Top-Down Production for Sentence Generation with Latent Trees [77.56794870399288]
  We model the production property of context-free grammars for natural and synthetic languages. We present a dynamic programming algorithm that marginalises over latent binary tree structures with $N$ leaves. We also present experimental results on German-English translation on the Multi30k dataset.
  arXiv  Detail & Related papers  (2020-10-09T17:47:16Z)
• Multitask Pointer Network for Multi-Representational Parsing [0.34376560669160383]
  We propose a transition-based approach that, by training a single model, can efficiently parse any input sentence with both constituent and dependency trees. We develop a Pointer Network architecture with two separate task-specific decoders and a common encoder, and follow a learning strategy to jointly train them.
  arXiv  Detail & Related papers  (2020-09-21T10:04:07Z)
• Efficient Constituency Parsing by Pointing [21.395573911155495]
  We propose a novel constituency parsing model that casts the parsing problem into a series of pointing tasks. Our model supports efficient top-down decoding and our learning objective is able to enforce structural consistency without resorting to the expensive CKY inference.
  arXiv  Detail & Related papers  (2020-06-24T08:29:09Z)
• Binarizing MobileNet via Evolution-based Searching [66.94247681870125]
  We propose a use of evolutionary search to facilitate the construction and training scheme when binarizing MobileNet. Inspired by one-shot architecture search frameworks, we manipulate the idea of group convolution to design efficient 1-Bit Convolutional Neural Networks (CNNs) Our objective is to come up with a tiny yet efficient binary neural architecture by exploring the best candidates of the group convolution.
  arXiv  Detail & Related papers  (2020-05-13T13:25:51Z)
• Exploiting Syntactic Structure for Better Language Modeling: A Syntactic Distance Approach [78.77265671634454]
  We make use of a multi-task objective, i.e., the models simultaneously predict words as well as ground truth parse trees in a form called "syntactic distances" Experimental results on the Penn Treebank and Chinese Treebank datasets show that when ground truth parse trees are provided as additional training signals, the model is able to achieve lower perplexity and induce trees with better quality.
  arXiv  Detail & Related papers  (2020-05-12T15:35:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.