Supertagging the Long Tail with Tree-Structured Decoding of Complex Categories

• URL: http://arxiv.org/abs/2012.01285v2
• Date: Fri, 11 Dec 2020 15:10:25 GMT
• Title: Supertagging the Long Tail with Tree-Structured Decoding of Complex Categories
• Authors: Jakob Prange, Nathan Schneider, Vivek Srikumar
• Abstract summary: Current CCG supertaggers achieve high accuracy on the standard WSJ test set, but few systems make use of the categories' internal structure. We investigate constructive models that account for their internal structure, including novel methods for tree-structured prediction. Our best tagger is capable of recovering a sizeable fraction of the long-tail supertags and even generates CCG categories that have never been seen in training.
• Score: 26.657488131046865
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Although current CCG supertaggers achieve high accuracy on the standard WSJ test set, few systems make use of the categories' internal structure that will drive the syntactic derivation during parsing. The tagset is traditionally truncated, discarding the many rare and complex category types in the long tail. However, supertags are themselves trees. Rather than give up on rare tags, we investigate constructive models that account for their internal structure, including novel methods for tree-structured prediction. Our best tagger is capable of recovering a sizeable fraction of the long-tail supertags and even generates CCG categories that have never been seen in training, while approximating the prior state of the art in overall tag accuracy with fewer parameters. We further investigate how well different approaches generalize to out-of-domain evaluation sets.

Related papers

• Integrating Supertag Features into Neural Discontinuous Constituent Parsing [0.0]
  Traditional views of constituency demand that constituents consist of adjacent words, common in languages like German. Transition-based parsing produces trees given raw text input using supervised learning on large annotated corpora.
  arXiv  Detail & Related papers  (2024-10-11T12:28:26Z)
• Grokking of Hierarchical Structure in Vanilla Transformers [72.45375959893218]
  We show that transformer language models can learn to generalize hierarchically after training for extremely long periods. intermediate-depth models generalize better than both very deep and very shallow transformers.
  arXiv  Detail & Related papers  (2023-05-30T04:34:13Z)
• Geometry-Aware Supertagging with Heterogeneous Dynamic Convolutions [0.7868449549351486]
  We revisit constructive supertagging from a graph-theoretic perspective. We propose a framework based on heterogeneous dynamic graph convolutions. We test our approach on a number of categorial grammar datasets spanning different languages.
  arXiv  Detail & Related papers  (2022-03-23T07:07:11Z)
• Compositional Generalization Requires Compositional Parsers [69.77216620997305]
  We compare sequence-to-sequence models and models guided by compositional principles on the recent COGS corpus. We show structural generalization is a key measure of compositional generalization and requires models that are aware of complex structure.
  arXiv  Detail & Related papers  (2022-02-24T07:36:35Z)
• Something Old, Something New: Grammar-based CCG Parsing with Transformer Models [13.035551917639692]
  This report describes the parsing problem for Combinatory Categorial Grammar (CCG) It shows how a combination of Transformer-based neural models and a symbolic CCG grammar can lead to substantial gains over existing approaches. It also documents a 20-year research program, showing how NLP methods have evolved over this time.
  arXiv  Detail & Related papers  (2021-09-21T09:20:12Z)
• Generating CCG Categories [20.154553201329712]
  We propose to generate categories rather than classify them. We show that with this finer view on categories, annotations of different categories could be shared and interactions with sentence contexts could be enhanced. The proposed category generator is able to achieve state-of-the-art tagging (95.5% accuracy) and parsing (89.8% labeled F1) performances on the standard CCGBank.
  arXiv  Detail & Related papers  (2021-03-15T05:01:48Z)
• Span-based Semantic Parsing for Compositional Generalization [53.24255235340056]
  SpanBasedSP predicts a span tree over an input utterance, explicitly encoding how partial programs compose over spans in the input. On GeoQuery, SCAN and CLOSURE, SpanBasedSP performs similarly to strong seq2seq baselines on random splits, but dramatically improves performance compared to baselines on splits that require compositional generalization.
  arXiv  Detail & Related papers  (2020-09-13T16:42:18Z)
• Forest R-CNN: Large-Vocabulary Long-Tailed Object Detection and Instance Segmentation [75.93960390191262]
  We exploit prior knowledge of the relations among object categories to cluster fine-grained classes into coarser parent classes. We propose a simple yet effective resampling method, NMS Resampling, to re-balance the data distribution. Our method, termed as Forest R-CNN, can serve as a plug-and-play module being applied to most object recognition models.
  arXiv  Detail & Related papers  (2020-08-13T03:52:37Z)
• Solving Long-tailed Recognition with Deep Realistic Taxonomic Classifier [68.38233199030908]
  Long-tail recognition tackles the natural non-uniformly distributed data in realworld scenarios. While moderns perform well on populated classes, its performance degrades significantly on tail classes. Deep-RTC is proposed as a new solution to the long-tail problem, combining realism with hierarchical predictions.
  arXiv  Detail & Related papers  (2020-07-20T05:57:42Z)
• Fine-Grained Visual Classification with Efficient End-to-end Localization [49.9887676289364]
  We present an efficient localization module that can be fused with a classification network in an end-to-end setup. We evaluate the new model on the three benchmark datasets CUB200-2011, Stanford Cars and FGVC-Aircraft.
  arXiv  Detail & Related papers  (2020-05-11T14:07:06Z)
• Hierarchical Entity Typing via Multi-level Learning to Rank [38.509244927293715]
  We propose a novel method for hierarchical entity classification that embraces ontological structure at both training and during prediction. At training, our novel multi-level learning-to-rank loss compares positive types against negative siblings according to the type tree. During prediction, we define a coarse-to-fine decoder that restricts viable candidates at each level of the ontology based on already predicted parent type(s)
  arXiv  Detail & Related papers  (2020-04-05T19:27:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.