Inducing and Using Alignments for Transition-based AMR Parsing

• URL: http://arxiv.org/abs/2205.01464v1
• Date: Tue, 3 May 2022 12:58:36 GMT
• Title: Inducing and Using Alignments for Transition-based AMR Parsing
• Authors: Andrew Drozdov, Jiawei Zhou, Radu Florian, Andrew McCallum, Tahira Naseem, Yoon Kim, Ramon Fernandez Astudillo
• Abstract summary: We propose a neural aligner for AMR that learns node-to-word alignments without relying on complex pipelines. We attain a new state-of-the art for gold-only trained models, matching silver-trained performance without the need for beam search on AMR3.0.
• Score: 51.35194383275297
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Transition-based parsers for Abstract Meaning Representation (AMR) rely on node-to-word alignments. These alignments are learned separately from parser training and require a complex pipeline of rule-based components, pre-processing, and post-processing to satisfy domain-specific constraints. Parsers also train on a point-estimate of the alignment pipeline, neglecting the uncertainty due to the inherent ambiguity of alignment. In this work we explore two avenues for overcoming these limitations. First, we propose a neural aligner for AMR that learns node-to-word alignments without relying on complex pipelines. We subsequently explore a tighter integration of aligner and parser training by considering a distribution over oracle action sequences arising from aligner uncertainty. Empirical results show this approach leads to more accurate alignments and generalization better from the AMR2.0 to AMR3.0 corpora. We attain a new state-of-the art for gold-only trained models, matching silver-trained performance without the need for beam search on AMR3.0.

Related papers

• On The Global Convergence Of Online RLHF With Neural Parametrization [36.239015146313136]
  Reinforcement Learning from Human Feedback (RLHF) aims to align large language models with human values. RLHF is a three-stage process that includes supervised fine-tuning, reward learning, and policy learning. We propose a bi-level formulation for AI alignment in parameterized settings and introduce a first-order approach to solve this problem.
  arXiv  Detail & Related papers  (2024-10-21T03:13:35Z)
• Beyond Imitation: Leveraging Fine-grained Quality Signals for Alignment [105.34140537748546]
  We propose an improved alignment approach named FIGA. Different from prior methods, we incorporate fine-grained quality signals that are derived by contrasting good and bad responses. Our approach has made two major contributions. Firstly, we curate a refined alignment dataset that pairs initial responses and the corresponding revised ones. Secondly, we devise a new loss function can leverage fine-grained quality signals to instruct the learning of LLMs for alignment.
  arXiv  Detail & Related papers  (2023-11-07T15:36:40Z)
• Stable Nonconvex-Nonconcave Training via Linear Interpolation [51.668052890249726]
  This paper presents a theoretical analysis of linearahead as a principled method for stabilizing (large-scale) neural network training. We argue that instabilities in the optimization process are often caused by the nonmonotonicity of the loss landscape and show how linear can help by leveraging the theory of nonexpansive operators.
  arXiv  Detail & Related papers  (2023-10-20T12:45:12Z)
• Guiding AMR Parsing with Reverse Graph Linearization [45.37129580211495]
  We propose a novel Reverse Graph Linearization (RGL) framework for AMR parsing. RGL defines both default and reverse linearization orders of an AMR graph, where most structures at the back part of the default order appear at the front part of the reversed order and vice versa. Our analysis shows that our proposed method significantly mitigates the problem of structure loss accumulation, outperforming the previously best AMR parsing model by 0.8 and 0.5 Smatch scores on the AMR 2.0 and AMR 3.0 dataset, respectively.
  arXiv  Detail & Related papers  (2023-10-13T05:03:13Z)
• Meta-Learning Adversarial Bandit Algorithms [55.72892209124227]
  We study online meta-learning with bandit feedback. We learn to tune online mirror descent generalization (OMD) with self-concordant barrier regularizers.
  arXiv  Detail & Related papers  (2023-07-05T13:52:10Z)
• BERM: Training the Balanced and Extractable Representation for Matching to Improve Generalization Ability of Dense Retrieval [54.66399120084227]
  We propose a novel method to improve the generalization of dense retrieval via capturing matching signal called BERM. Dense retrieval has shown promise in the first-stage retrieval process when trained on in-domain labeled datasets.
  arXiv  Detail & Related papers  (2023-05-18T15:43:09Z)
• Probabilistic, Structure-Aware Algorithms for Improved Variety, Accuracy, and Coverage of AMR Alignments [9.74672460306765]
  We present algorithms for aligning components of Abstract Meaning Representation (AMR) spans in English sentences. We leverage unsupervised learning in combination with graphs, taking the best of both worlds from previous AMR. Our approach covers a wider variety of AMR substructures than previously considered, achieves higher coverage of nodes and edges, and does so with higher accuracy.
  arXiv  Detail & Related papers  (2021-06-10T18:46:32Z)
• A Differentiable Relaxation of Graph Segmentation and Alignment for AMR Parsing [75.36126971685034]
  We treat alignment and segmentation as latent variables in our model and induce them as part of end-to-end training. Our method also approaches that of a model that relies on citetLyu2018AMRPA's segmentation rules, which were hand-crafted to handle individual AMR constructions.
  arXiv  Detail & Related papers  (2020-10-23T21:22:50Z)
• Accurate Word Alignment Induction from Neural Machine Translation [33.21196289328584]
  We propose two novel word alignment induction methods Shift-Att and Shift-AET. The main idea is to induce alignments at the step when the to-be-aligned target token is the decoder input. Experiments on three publicly available datasets demonstrate that both methods perform better than their corresponding neural baselines.
  arXiv  Detail & Related papers  (2020-04-30T14:47:05Z)
• Stratified Rule-Aware Network for Abstract Visual Reasoning [46.015682319351676]
  Raven's Progressive Matrices (RPM) test is typically used to examine the capability of abstract reasoning. Recent studies, taking advantage of Convolutional Neural Networks (CNNs), have achieved encouraging progress to accomplish the RPM test. We propose a Stratified Rule-Aware Network (SRAN) to generate the rule embeddings for two input sequences.
  arXiv  Detail & Related papers  (2020-02-17T08:44:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.