AMPERE: AMR-Aware Prefix for Generation-Based Event Argument Extraction Model

• URL: http://arxiv.org/abs/2305.16734v1
• Date: Fri, 26 May 2023 08:38:25 GMT
• Title: AMPERE: AMR-Aware Prefix for Generation-Based Event Argument Extraction Model
• Authors: I-Hung Hsu, Zhiyu Xie, Kuan-Hao Huang, Prem Natarajan, Nanyun Peng
• Abstract summary: Event argument extraction (EAE) identifies event arguments and their specific roles for a given event. Recent advancement in generation-based EAE models has shown great performance and generalizability over classification-based models. We propose AMPERE, which generates AMR-aware prefixes for every layer of the generation model.
• Score: 38.390078345679214
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Event argument extraction (EAE) identifies event arguments and their specific roles for a given event. Recent advancement in generation-based EAE models has shown great performance and generalizability over classification-based models. However, existing generation-based EAE models mostly focus on problem re-formulation and prompt design, without incorporating additional information that has been shown to be effective for classification-based models, such as the abstract meaning representation (AMR) of the input passages. Incorporating such information into generation-based models is challenging due to the heterogeneous nature of the natural language form prevalently used in generation-based models and the structured form of AMRs. In this work, we study strategies to incorporate AMR into generation-based EAE models. We propose AMPERE, which generates AMR-aware prefixes for every layer of the generation model. Thus, the prefix introduces AMR information to the generation-based EAE model and then improves the generation. We also introduce an adjusted copy mechanism to AMPERE to help overcome potential noises brought by the AMR graph. Comprehensive experiments and analyses on ACE2005 and ERE datasets show that AMPERE can get 4% - 10% absolute F1 score improvements with reduced training data and it is in general powerful across different training sizes.

Related papers

• Steering Masked Discrete Diffusion Models via Discrete Denoising Posterior Prediction [88.65168366064061]
  We introduce Discrete Denoising Posterior Prediction (DDPP), a novel framework that casts the task of steering pre-trained MDMs as a problem of probabilistic inference. Our framework leads to a family of three novel objectives that are all simulation-free, and thus scalable. We substantiate our designs via wet-lab validation, where we observe transient expression of reward-optimized protein sequences.
  arXiv  Detail & Related papers  (2024-10-10T17:18:30Z)
• A Structure-aware Generative Model for Biomedical Event Extraction [6.282854894433099]
  Event structure-aware generative model named GenBEE can capture complex event structures in biomedical text. We have evaluated the proposed GenBEE model on three widely used biomedical event extraction benchmark datasets.
  arXiv  Detail & Related papers  (2024-08-13T02:43:19Z)
• Synthesizing Multimodal Electronic Health Records via Predictive Diffusion Models [69.06149482021071]
  We propose a novel EHR data generation model called EHRPD. It is a diffusion-based model designed to predict the next visit based on the current one while also incorporating time interval estimation. We conduct experiments on two public datasets and evaluate EHRPD from fidelity, privacy, and utility perspectives.
  arXiv  Detail & Related papers  (2024-06-20T02:20:23Z)
• Comprehensive Exploration of Synthetic Data Generation: A Survey [4.485401662312072]
  This work surveys 417 Synthetic Data Generation models over the last decade. The findings reveal increased model performance and complexity, with neural network-based approaches prevailing. Computer vision dominates, with GANs as primary generative models, while diffusion models, transformers, and RNNs compete.
  arXiv  Detail & Related papers  (2024-01-04T20:23:51Z)
• AMR Parsing with Causal Hierarchical Attention and Pointers [54.382865897298046]
  We introduce new target forms of AMR parsing and a novel model, CHAP, which is equipped with causal hierarchical attention and the pointer mechanism. Experiments show that our model outperforms baseline models on four out of five benchmarks in the setting of no additional data.
  arXiv  Detail & Related papers  (2023-10-18T13:44:26Z)
• A Probabilistic Fluctuation based Membership Inference Attack for Diffusion Models [32.15773300068426]
  Membership Inference Attack (MIA) identifies whether a record exists in a machine learning model's training set by querying the model. We propose a Probabilistic Fluctuation Assessing Membership Inference Attack (PFAMI) PFAMI can improve the attack success rate (ASR) by about 27.9% when compared with the best baseline.
  arXiv  Detail & Related papers  (2023-08-23T14:00:58Z)
• End-to-End Speech Recognition: A Survey [68.35707678386949]
  The goal of this survey is to provide a taxonomy of E2E ASR models and corresponding improvements. All relevant aspects of E2E ASR are covered in this work, accompanied by discussions of performance and deployment opportunities.
  arXiv  Detail & Related papers  (2023-03-03T01:46:41Z)
• GENEVA: Benchmarking Generalizability for Event Argument Extraction with Hundreds of Event Types and Argument Roles [77.05288144035056]
  Event Argument Extraction (EAE) has focused on improving model generalizability to cater to new events and domains. Standard benchmarking datasets like ACE and ERE cover less than 40 event types and 25 entity-centric argument roles.
  arXiv  Detail & Related papers  (2022-05-25T05:46:28Z)
• Reconstructing Training Data from Diverse ML Models by Ensemble Inversion [8.414622657659168]
  Model Inversion (MI), in which an adversary abuses access to a trained Machine Learning (ML) model, has attracted increasing research attention. We propose an ensemble inversion technique that estimates the distribution of original training data by training a generator constrained by an ensemble of trained models. We achieve high quality results without any dataset and show how utilizing an auxiliary dataset that's similar to the presumed training data improves the results.
  arXiv  Detail & Related papers  (2021-11-05T18:59:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.