Order-sensitive Neural Constituency Parsing

• URL: http://arxiv.org/abs/2211.00421v1
• Date: Tue, 1 Nov 2022 12:31:30 GMT
• Title: Order-sensitive Neural Constituency Parsing
• Authors: Zhicheng Wang, Tianyu Shi, Liyin Xiao, Cong Liu
• Abstract summary: We propose a novel algorithm that improves on the previous neural span-based CKY decoder for constituency parsing. In contrast to the traditional span-based decoding, we introduce an order-sensitive strategy, where the span combination scores are more carefully derived from an order-sensitive basis. Our decoder can be regarded as a generalization over existing span-based decoder in determining a finer-grain scoring scheme for the combination of lower-level spans into higher-level spans.
• Score: 9.858565876426411
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a novel algorithm that improves on the previous neural span-based CKY decoder for constituency parsing. In contrast to the traditional span-based decoding, where spans are combined only based on the sum of their scores, we introduce an order-sensitive strategy, where the span combination scores are more carefully derived from an order-sensitive basis. Our decoder can be regarded as a generalization over existing span-based decoder in determining a finer-grain scoring scheme for the combination of lower-level spans into higher-level spans, where we emphasize on the order of the lower-level spans and use order-sensitive span scores as well as order-sensitive combination grammar rule scores to enhance prediction accuracy. We implement the proposed decoding strategy harnessing GPU parallelism and achieve a decoding speed on par with state-of-the-art span-based parsers. Using the previous state-of-the-art model without additional data as our baseline, we outperform it and improve the F1 score on the Penn Treebank Dataset by 0.26% and on the Chinese Treebank Dataset by 0.35%.

Related papers

• Learning-to-Rank Meets Language: Boosting Language-Driven Ordering Alignment for Ordinal Classification [60.28913031192201]
  We present a novel language-driven ordering alignment method for ordinal classification. Recent developments in pre-trained vision-language models inspire us to leverage the rich ordinal priors in human language. Experiments on three ordinal classification tasks, including facial age estimation, historical color image (HCI) classification, and aesthetic assessment demonstrate its promising performance.
  arXiv  Detail & Related papers  (2023-06-24T04:11:31Z)
• Scalable Learning of Latent Language Structure With Logical Offline Cycle Consistency [71.42261918225773]
  Conceptually, LOCCO can be viewed as a form of self-learning where the semantic being trained is used to generate annotations for unlabeled text. As an added bonus, the annotations produced by LOCCO can be trivially repurposed to train a neural text generation model.
  arXiv  Detail & Related papers  (2023-05-31T16:47:20Z)
• Fast Rule-Based Decoding: Revisiting Syntactic Rules in Neural Constituency Parsing [9.858565876426411]
  Previous research has demonstrated that probabilistic statistical methods based on syntactic rules are particularly effective in constituency parsing. In this paper, we first implement a fast CKY decoding procedure harnessing GPU acceleration, based on which we further derive a syntactic rule-based (rule-constrained) CKY decoding.
  arXiv  Detail & Related papers  (2022-12-16T13:07:09Z)
• A Character-level Span-based Model for Mandarin Prosodic Structure Prediction [36.90699361223442]
  We propose a span-based Mandarin prosodic structure prediction model to obtain an optimal prosodic structure tree. Rich linguistic features are provided by Chinese character-level BERT and sent to encoder with self-attention architecture. The proposed method can predict prosodic labels of different levels at the same time and accomplish the process directly from Chinese characters.
  arXiv  Detail & Related papers  (2022-03-31T09:47:08Z)
• Speaker Embedding-aware Neural Diarization: a Novel Framework for Overlapped Speech Diarization in the Meeting Scenario [51.5031673695118]
  We reformulate overlapped speech diarization as a single-label prediction problem. We propose the speaker embedding-aware neural diarization (SEND) system.
  arXiv  Detail & Related papers  (2022-03-18T06:40:39Z)
• GNNRank: Learning Global Rankings from Pairwise Comparisons via Directed Graph Neural Networks [68.61934077627085]
  We introduce GNNRank, a modeling framework compatible with any GNN capable of learning digraph embeddings. We show that our methods attain competitive and often superior performance compared with existing approaches.
  arXiv  Detail & Related papers  (2022-02-01T04:19:50Z)
• Reinforcement Learning Based Query Vertex Ordering Model for Subgraph Matching [58.39970828272366]
  Subgraph matching algorithms enumerate all is embeddings of a query graph in a data graph G. matching order plays a critical role in time efficiency of these backtracking based subgraph matching algorithms. In this paper, for the first time we apply the Reinforcement Learning (RL) and Graph Neural Networks (GNNs) techniques to generate the high-quality matching order for subgraph matching algorithms.
  arXiv  Detail & Related papers  (2022-01-25T00:10:03Z)
• Headed Span-Based Projective Dependency Parsing [24.337440797369702]
  We propose a headed span-based method for projective dependency parsing. We use neural networks to score headed spans and design a novel $O(n3)$ dynamic programming algorithm to enable global training and exact inference.
  arXiv  Detail & Related papers  (2021-08-10T15:27:47Z)
• Deep Diacritization: Efficient Hierarchical Recurrence for Improved Arabic Diacritization [0.0]
  We propose a novel architecture for labelling character sequences that achieves state-of-the-art results on the Tashkeela Arabic diacritization benchmark. The core is a two-level recurrence hierarchy that operates on the word and character levels separately. A cross-level attention module further connects the two, and opens the door for network interpretability.
  arXiv  Detail & Related papers  (2020-11-01T15:33:43Z)
• Span-based Semantic Parsing for Compositional Generalization [53.24255235340056]
  SpanBasedSP predicts a span tree over an input utterance, explicitly encoding how partial programs compose over spans in the input. On GeoQuery, SCAN and CLOSURE, SpanBasedSP performs similarly to strong seq2seq baselines on random splits, but dramatically improves performance compared to baselines on splits that require compositional generalization.
  arXiv  Detail & Related papers  (2020-09-13T16:42:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.