TreePiece: Faster Semantic Parsing via Tree Tokenization

• URL: http://arxiv.org/abs/2303.17161v1
• Date: Thu, 30 Mar 2023 05:44:44 GMT
• Title: TreePiece: Faster Semantic Parsing via Tree Tokenization
• Authors: Sid Wang, Akshat Shrivastava, Sasha Livshits
• Abstract summary: TreePiece tokenizes a parse tree into subtrees and generates one subtree per decoding step. On TopV2 benchmark, TreePiece shows 4.6 times faster decoding speed than standard AR.
• Score: 2.1685554819849613
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Autoregressive (AR) encoder-decoder neural networks have proved successful in many NLP problems, including Semantic Parsing -- a task that translates natural language to machine-readable parse trees. However, the sequential prediction process of AR models can be slow. To accelerate AR for semantic parsing, we introduce a new technique called TreePiece that tokenizes a parse tree into subtrees and generates one subtree per decoding step. On TopV2 benchmark, TreePiece shows 4.6 times faster decoding speed than standard AR, and comparable speed but significantly higher accuracy compared to Non-Autoregressive (NAR).

Related papers

• Context Perception Parallel Decoder for Scene Text Recognition [52.620841341333524]
  Scene text recognition methods have struggled to attain high accuracy and fast inference speed. We present an empirical study of AR decoding in STR, and discover that the AR decoder not only models linguistic context, but also provides guidance on visual context perception. We construct a series of CPPD models and also plug the proposed modules into existing STR decoders. Experiments on both English and Chinese benchmarks demonstrate that the CPPD models achieve highly competitive accuracy while running approximately 8x faster than their AR-based counterparts.
  arXiv  Detail & Related papers  (2023-07-23T09:04:13Z)
• Hexatagging: Projective Dependency Parsing as Tagging [63.5392760743851]
  We introduce a novel dependency, the hexatagger, that constructs dependency trees by tagging the words in a sentence with elements from a finite set of possible tags. Our approach is fully parallelizable at training time, i.e., the structure-building actions needed to build a dependency parse can be predicted in parallel to each other. We achieve state-of-the-art performance of 96.4 LAS and 97.4 UAS on the Penn Treebank test set.
  arXiv  Detail & Related papers  (2023-06-08T18:02:07Z)
• Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognition [62.83832841523525]
  We propose a fast and accurate parallel transformer, termed Paraformer. It accurately predicts the number of output tokens and extract hidden variables. It can attain comparable performance to the state-of-the-art AR transformer, with more than 10x speedup.
  arXiv  Detail & Related papers  (2022-06-16T17:24:14Z)
• Fast-R2D2: A Pretrained Recursive Neural Network based on Pruned CKY for Grammar Induction and Text Representation [41.51966652141165]
  We propose a model-based pruning method, which also enables parallel encoding during inference. Our experiments show that our Fast-R2D2 improves performance significantly in grammar induction and competitive results in downstream classification tasks.
  arXiv  Detail & Related papers  (2022-03-01T07:54:44Z)
• Recursive Tree Grammar Autoencoders [3.791857415239352]
  We propose a novel autoencoder approach that encodes trees via a bottom-up grammar and decodes trees via a tree grammar. We show experimentally that our proposed method improves the autoencoding error, training time, and optimization score on four benchmark datasets.
  arXiv  Detail & Related papers  (2020-12-03T17:37:25Z)
• Strongly Incremental Constituency Parsing with Graph Neural Networks [70.16880251349093]
  Parsing sentences into syntax trees can benefit downstream applications in NLP. Transition-baseds build trees by executing actions in a state transition system. Existing transition-baseds are predominantly based on the shift-reduce transition system.
  arXiv  Detail & Related papers  (2020-10-27T19:19:38Z)
• SmBoP: Semi-autoregressive Bottom-up Semantic Parsing [44.802643057976354]
  We propose a Semi-autoregressive Bottom-up (SmBoP) that constructs at decoding step $t$ the top-$K$ sub-trees of height $leq t$. From an efficiency perspective, bottom-up parsing allows to decode all sub-trees of certain height in parallel, leading to logarithmic complexity runtime rather than linear. We apply SmBoP on Spider, a challenging zero-shot semantic parsing benchmark, and show that SmBoP leads to a 2.2x speed-up in decoding time and a $$5x speed-up in training time
  arXiv  Detail & Related papers  (2020-10-23T14:02:32Z)
• Fast semantic parsing with well-typedness guarantees [78.76675218975768]
  AM dependency parsing is a principled method for neural semantic parsing with high accuracy across multiple graphbanks. We describe an A* and a transition-based for AM dependency parsing which guarantee well-typedness and improve parsing speed by up to 3 orders of magnitude.
  arXiv  Detail & Related papers  (2020-09-15T21:54:01Z)
• Efficient Second-Order TreeCRF for Neural Dependency Parsing [23.426500262860777]
  In the deep learning (DL) era, parsing models are extremely simplified with little hurt on performance. This paper presents a second-order TreeCRF extension to the biaffine. We propose an effective way to batchify the inside and Viterbi algorithms for direct large matrix operation.
  arXiv  Detail & Related papers  (2020-05-03T03:18:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.