Translate First Reorder Later: Leveraging Monotonicity in Semantic Parsing

• URL: http://arxiv.org/abs/2210.04878v1
• Date: Mon, 10 Oct 2022 17:50:42 GMT
• Title: Translate First Reorder Later: Leveraging Monotonicity in Semantic Parsing
• Authors: Francesco Cazzaro, Davide Locatelli, Ariadna Quattoni, Xavier Carreras
• Abstract summary: TPol is a two-step approach that translates input sentences monotonically and then reorders them to obtain the correct output. We test our approach on two popular semantic parsing datasets.
• Score: 4.396860522241306
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Prior work in semantic parsing has shown that conventional seq2seq models fail at compositional generalization tasks. This limitation led to a resurgence of methods that model alignments between sentences and their corresponding meaning representations, either implicitly through latent variables or explicitly by taking advantage of alignment annotations. We take the second direction and propose TPol, a two-step approach that first translates input sentences monotonically and then reorders them to obtain the correct output. This is achieved with a modular framework comprising a Translator and a Reorderer component. We test our approach on two popular semantic parsing datasets. Our experiments show that by means of the monotonic translations, TPol can learn reliable lexico-logical patterns from aligned data, significantly improving compositional generalization both over conventional seq2seq models, as well as over a recently proposed approach that exploits gold alignments.

Related papers

• Improving Generalization in Language Model-Based Text-to-SQL Semantic Parsing: Two Simple Semantic Boundary-Based Techniques [14.634536051274468]
  We introduce a token preprocessing method to preserve the semantic boundaries of tokens produced by LM tokenizers. At the sequence level, we propose to use special tokens to mark the boundaries of components aligned between input and output. Our experimental results on two text-to- semantic parsing datasets show that our token preprocessing, although simple, can substantially improve the LM performance on both types of generalization.
  arXiv  Detail & Related papers  (2023-05-27T06:09:03Z)
• Compositional Generalization without Trees using Multiset Tagging and Latent Permutations [121.37328648951993]
  We phrase semantic parsing as a two-step process: we first tag each input token with a multiset of output tokens. Then we arrange the tokens into an output sequence using a new way of parameterizing and predicting permutations. Our model outperforms pretrained seq2seq models and prior work on realistic semantic parsing tasks.
  arXiv  Detail & Related papers  (2023-05-26T14:09:35Z)
• Mutual Exclusivity Training and Primitive Augmentation to Induce Compositionality [84.94877848357896]
  Recent datasets expose the lack of the systematic generalization ability in standard sequence-to-sequence models. We analyze this behavior of seq2seq models and identify two contributing factors: a lack of mutual exclusivity bias and the tendency to memorize whole examples. We show substantial empirical improvements using standard sequence-to-sequence models on two widely-used compositionality datasets.
  arXiv  Detail & Related papers  (2022-11-28T17:36:41Z)
• Hierarchical Phrase-based Sequence-to-Sequence Learning [94.10257313923478]
  We describe a neural transducer that maintains the flexibility of standard sequence-to-sequence (seq2seq) models while incorporating hierarchical phrases as a source of inductive bias during training and as explicit constraints during inference. Our approach trains two models: a discriminative derivation based on a bracketing grammar whose tree hierarchically aligns source and target phrases, and a neural seq2seq model that learns to translate the aligned phrases one-by-one.
  arXiv  Detail & Related papers  (2022-11-15T05:22:40Z)
• Measuring Alignment Bias in Neural Seq2Seq Semantic Parsers [2.8681767712152193]
  We investigate whether seq2seq models can handle both simple and complex alignments. Our empirical study shows that performance is significantly better over monotonic alignments.
  arXiv  Detail & Related papers  (2022-05-17T12:35:52Z)
• Grounded Graph Decoding Improves Compositional Generalization in Question Answering [68.72605660152101]
  Question answering models struggle to generalize to novel compositions of training patterns, such as longer sequences or more complex test structures. We propose Grounded Graph Decoding, a method to improve compositional generalization of language representations by grounding structured predictions with an attention mechanism. Our model significantly outperforms state-of-the-art baselines on the Compositional Freebase Questions (CFQ) dataset, a challenging benchmark for compositional generalization in question answering.
  arXiv  Detail & Related papers  (2021-11-05T17:50:14Z)
• Structured Reordering for Modeling Latent Alignments in Sequence Transduction [86.94309120789396]
  We present an efficient dynamic programming algorithm performing exact marginal inference of separable permutations. The resulting seq2seq model exhibits better systematic generalization than standard models on synthetic problems and NLP tasks.
  arXiv  Detail & Related papers  (2021-06-06T21:53:54Z)
• Compositional Generalization via Semantic Tagging [81.24269148865555]
  We propose a new decoding framework that preserves the expressivity and generality of sequence-to-sequence models. We show that the proposed approach consistently improves compositional generalization across model architectures, domains, and semantic formalisms.
  arXiv  Detail & Related papers  (2020-10-22T15:55:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.