Span-based Semantic Parsing for Compositional Generalization

• URL: http://arxiv.org/abs/2009.06040v2
• Date: Sun, 13 Jun 2021 09:07:23 GMT
• Title: Span-based Semantic Parsing for Compositional Generalization
• Authors: Jonathan Herzig and Jonathan Berant
• Abstract summary: SpanBasedSP predicts a span tree over an input utterance, explicitly encoding how partial programs compose over spans in the input. On GeoQuery, SCAN and CLOSURE, SpanBasedSP performs similarly to strong seq2seq baselines on random splits, but dramatically improves performance compared to baselines on splits that require compositional generalization.
• Score: 53.24255235340056
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite the success of sequence-to-sequence (seq2seq) models in semantic parsing, recent work has shown that they fail in compositional generalization, i.e., the ability to generalize to new structures built of components observed during training. In this work, we posit that a span-based parser should lead to better compositional generalization. we propose SpanBasedSP, a parser that predicts a span tree over an input utterance, explicitly encoding how partial programs compose over spans in the input. SpanBasedSP extends Pasupat et al. (2019) to be comparable to seq2seq models by (i) training from programs, without access to gold trees, treating trees as latent variables, (ii) parsing a class of non-projective trees through an extension to standard CKY. On GeoQuery, SCAN and CLOSURE datasets, SpanBasedSP performs similarly to strong seq2seq baselines on random splits, but dramatically improves performance compared to baselines on splits that require compositional generalization: from $61.0 \rightarrow 88.9$ average accuracy.

Related papers

• High-order Joint Constituency and Dependency Parsing [15.697429723696011]
  We revisit the topic of jointly parsing constituency and dependency trees, i.e., to produce compatible constituency and dependency trees simultaneously for input sentences. We conduct experiments and analysis on seven languages, covering both rich-resource and low-resource scenarios.
  arXiv  Detail & Related papers  (2023-09-21T08:45:41Z)
• Compositional Generalization without Trees using Multiset Tagging and Latent Permutations [121.37328648951993]
  We phrase semantic parsing as a two-step process: we first tag each input token with a multiset of output tokens. Then we arrange the tokens into an output sequence using a new way of parameterizing and predicting permutations. Our model outperforms pretrained seq2seq models and prior work on realistic semantic parsing tasks.
  arXiv  Detail & Related papers  (2023-05-26T14:09:35Z)
• Hierarchical Phrase-based Sequence-to-Sequence Learning [94.10257313923478]
  We describe a neural transducer that maintains the flexibility of standard sequence-to-sequence (seq2seq) models while incorporating hierarchical phrases as a source of inductive bias during training and as explicit constraints during inference. Our approach trains two models: a discriminative derivation based on a bracketing grammar whose tree hierarchically aligns source and target phrases, and a neural seq2seq model that learns to translate the aligned phrases one-by-one.
  arXiv  Detail & Related papers  (2022-11-15T05:22:40Z)
• Unsupervised and Few-shot Parsing from Pretrained Language Models [56.33247845224995]
  We propose an Unsupervised constituent Parsing model that calculates an Out Association score solely based on the self-attention weight matrix learned in a pretrained language model. We extend the unsupervised models to few-shot parsing models that use a few annotated trees to learn better linear projection matrices for parsing. Our few-shot parsing model FPIO trained with only 20 annotated trees outperforms a previous few-shot parsing method trained with 50 annotated trees.
  arXiv  Detail & Related papers  (2022-06-10T10:29:15Z)
• Compositional Task-Oriented Parsing as Abstractive Question Answering [25.682923914685063]
  Task-oriented parsing aims to convert natural language into machine-readable representations of specific tasks, such as setting an alarm. A popular approach to TOP is to apply seq2seq models to generate linearized parse trees. A more recent line of work argues that pretrained seq2seq models are better at generating outputs that are themselves natural language, so they replace linearized parse trees with canonical natural-language paraphrases.
  arXiv  Detail & Related papers  (2022-05-04T14:01:08Z)
• A Character-level Span-based Model for Mandarin Prosodic Structure Prediction [36.90699361223442]
  We propose a span-based Mandarin prosodic structure prediction model to obtain an optimal prosodic structure tree. Rich linguistic features are provided by Chinese character-level BERT and sent to encoder with self-attention architecture. The proposed method can predict prosodic labels of different levels at the same time and accomplish the process directly from Chinese characters.
  arXiv  Detail & Related papers  (2022-03-31T09:47:08Z)
• Compositional Generalization Requires Compositional Parsers [69.77216620997305]
  We compare sequence-to-sequence models and models guided by compositional principles on the recent COGS corpus. We show structural generalization is a key measure of compositional generalization and requires models that are aware of complex structure.
  arXiv  Detail & Related papers  (2022-02-24T07:36:35Z)
• Grounded Graph Decoding Improves Compositional Generalization in Question Answering [68.72605660152101]
  Question answering models struggle to generalize to novel compositions of training patterns, such as longer sequences or more complex test structures. We propose Grounded Graph Decoding, a method to improve compositional generalization of language representations by grounding structured predictions with an attention mechanism. Our model significantly outperforms state-of-the-art baselines on the Compositional Freebase Questions (CFQ) dataset, a challenging benchmark for compositional generalization in question answering.
  arXiv  Detail & Related papers  (2021-11-05T17:50:14Z)
• Headed Span-Based Projective Dependency Parsing [24.337440797369702]
  We propose a headed span-based method for projective dependency parsing. We use neural networks to score headed spans and design a novel $O(n3)$ dynamic programming algorithm to enable global training and exact inference.
  arXiv  Detail & Related papers  (2021-08-10T15:27:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.