Entity Aware Syntax Tree Based Data Augmentation for Natural Language Understanding

• URL: http://arxiv.org/abs/2209.02267v1
• Date: Tue, 6 Sep 2022 07:34:10 GMT
• Title: Entity Aware Syntax Tree Based Data Augmentation for Natural Language Understanding
• Authors: Jiaxing Xu, Jianbin Cui, Jiangneng Li, Wenge Rong and Noboru Matsuda
• Abstract summary: We propose a novel NLP data augmentation technique, which applies a tree structure, Entity Aware Syntax Tree (EAST) to represent sentences combined with attention on the entity. Our EADA technique automatically constructs an EAST from a small amount of annotated data, and then generates a large number of training instances for intent detection and slot filling. Experimental results on four datasets showed that the proposed technique significantly outperforms the existing data augmentation methods in terms of both accuracy and generalization ability.
• Score: 5.02493891738617
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Understanding the intention of the users and recognizing the semantic entities from their sentences, aka natural language understanding (NLU), is the upstream task of many natural language processing tasks. One of the main challenges is to collect a sufficient amount of annotated data to train a model. Existing research about text augmentation does not abundantly consider entity and thus performs badly for NLU tasks. To solve this problem, we propose a novel NLP data augmentation technique, Entity Aware Data Augmentation (EADA), which applies a tree structure, Entity Aware Syntax Tree (EAST), to represent sentences combined with attention on the entity. Our EADA technique automatically constructs an EAST from a small amount of annotated data, and then generates a large number of training instances for intent detection and slot filling. Experimental results on four datasets showed that the proposed technique significantly outperforms the existing data augmentation methods in terms of both accuracy and generalization ability.

Related papers

• Web-Scale Visual Entity Recognition: An LLM-Driven Data Approach [56.55633052479446]
  Web-scale visual entity recognition presents significant challenges due to the lack of clean, large-scale training data. We propose a novel methodology to curate such a dataset, leveraging a multimodal large language model (LLM) for label verification, metadata generation, and rationale explanation. Experiments demonstrate that models trained on this automatically curated data achieve state-of-the-art performance on web-scale visual entity recognition tasks.
  arXiv  Detail & Related papers  (2024-10-31T06:55:24Z)
• LaiDA: Linguistics-aware In-context Learning with Data Augmentation for Metaphor Components Identification [0.07989135005592125]
  Large language models (LLMs) offer new avenues for accurate comprehension of complex natural language texts. A new LLM-based framework is proposed, named Linguistics-aware In-context Learning with Data Augmentation (LaiDA) LaiDA incorporates a simile dataset for pre-training. A graph attention network encoder generates linguistically rich feature representations to retrieve similar examples.
  arXiv  Detail & Related papers  (2024-08-10T02:02:26Z)
• GDA: Generative Data Augmentation Techniques for Relation Extraction Tasks [81.51314139202152]
  We propose a dedicated augmentation technique for relational texts, named GDA, which uses two complementary modules to preserve both semantic consistency and syntax structures. Experimental results in three datasets under a low-resource setting showed that GDA could bring em 2.0% F1 improvements compared with no augmentation technique.
  arXiv  Detail & Related papers  (2023-05-26T06:21:01Z)
• STAR: Boosting Low-Resource Information Extraction by Structure-to-Text Data Generation with Large Language Models [56.27786433792638]
  STAR is a data generation method that leverages Large Language Models (LLMs) to synthesize data instances. We design fine-grained step-by-step instructions to obtain the initial data instances. Our experiments show that the data generated by STAR significantly improve the performance of low-resource event extraction and relation extraction tasks.
  arXiv  Detail & Related papers  (2023-05-24T12:15:19Z)
• Boosting Event Extraction with Denoised Structure-to-Text Augmentation [52.21703002404442]
  Event extraction aims to recognize pre-defined event triggers and arguments from texts. Recent data augmentation methods often neglect the problem of grammatical incorrectness. We propose a denoised structure-to-text augmentation framework for event extraction DAEE.
  arXiv  Detail & Related papers  (2023-05-16T16:52:07Z)
• Unified BERT for Few-shot Natural Language Understanding [7.352338840651369]
  We propose UBERT, a unified bidirectional language understanding model based on BERT framework. UBERT encodes prior knowledge from various aspects, uniformly constructing learning representations across multiple NLU tasks. Experiments show that UBERT achieves the state-of-the-art performance on 7 NLU tasks, 14 datasets on few-shot and zero-shot setting.
  arXiv  Detail & Related papers  (2022-06-24T06:10:53Z)
• On the Use of External Data for Spoken Named Entity Recognition [40.93448412171246]
  Recent advances in self-supervised speech representations have made it feasible to consider learning models with limited labeled data. We draw on a variety of approaches, including self-training, knowledge distillation, and transfer learning, and consider their applicability to both end-to-end models and pipeline approaches.
  arXiv  Detail & Related papers  (2021-12-14T18:49:26Z)
• Structured Prediction as Translation between Augmented Natural Languages [109.50236248762877]
  We propose a new framework, Translation between Augmented Natural Languages (TANL), to solve many structured prediction language tasks. Instead of tackling the problem by training task-specific discriminatives, we frame it as a translation task between augmented natural languages. Our approach can match or outperform task-specific models on all tasks, and in particular, achieves new state-of-the-art results on joint entity and relation extraction.
  arXiv  Detail & Related papers  (2021-01-14T18:32:21Z)
• LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention [37.111204321059084]
  We propose new pretrained contextualized representations of words and entities based on the bidirectional transformer. Our model is trained using a new pretraining task based on the masked language model of BERT. We also propose an entity-aware self-attention mechanism that is an extension of the self-attention mechanism of the transformer.
  arXiv  Detail & Related papers  (2020-10-02T15:38:03Z)
• Probing Linguistic Features of Sentence-Level Representations in Neural Relation Extraction [80.38130122127882]
  We introduce 14 probing tasks targeting linguistic properties relevant to neural relation extraction (RE) We use them to study representations learned by more than 40 different encoder architecture and linguistic feature combinations trained on two datasets. We find that the bias induced by the architecture and the inclusion of linguistic features are clearly expressed in the probing task performance.
  arXiv  Detail & Related papers  (2020-04-17T09:17:40Z)
• Hybrid Attention-Based Transformer Block Model for Distant Supervision Relation Extraction [20.644215991166902]
  We propose a new framework using hybrid attention-based Transformer block with multi-instance learning to perform the DSRE task. The proposed approach can outperform the state-of-the-art algorithms on the evaluation dataset.
  arXiv  Detail & Related papers  (2020-03-10T13:05:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.