ChatGPT as Data Augmentation for Compositional Generalization: A Case Study in Open Intent Detection

• URL: http://arxiv.org/abs/2308.13517v1
• Date: Fri, 25 Aug 2023 17:51:23 GMT
• Title: ChatGPT as Data Augmentation for Compositional Generalization: A Case Study in Open Intent Detection
• Authors: Yihao Fang, Xianzhi Li, Stephen W. Thomas, Xiaodan Zhu
• Abstract summary: We present a case study exploring the use of ChatGPT as a data augmentation technique to enhance compositional generalization in open intent detection tasks. By incorporating synthetic data generated by ChatGPT into the training process, we demonstrate that our approach can effectively improve model performance.
• Score: 30.13634341221476
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Open intent detection, a crucial aspect of natural language understanding, involves the identification of previously unseen intents in user-generated text. Despite the progress made in this field, challenges persist in handling new combinations of language components, which is essential for compositional generalization. In this paper, we present a case study exploring the use of ChatGPT as a data augmentation technique to enhance compositional generalization in open intent detection tasks. We begin by discussing the limitations of existing benchmarks in evaluating this problem, highlighting the need for constructing datasets for addressing compositional generalization in open intent detection tasks. By incorporating synthetic data generated by ChatGPT into the training process, we demonstrate that our approach can effectively improve model performance. Rigorous evaluation of multiple benchmarks reveals that our method outperforms existing techniques and significantly enhances open intent detection capabilities. Our findings underscore the potential of large language models like ChatGPT for data augmentation in natural language understanding tasks.

Related papers

• Learning Task Representations from In-Context Learning [73.72066284711462]
  Large language models (LLMs) have demonstrated remarkable proficiency in in-context learning. We introduce an automated formulation for encoding task information in ICL prompts as a function of attention heads. We show that our method's effectiveness stems from aligning the distribution of the last hidden state with that of an optimally performing in-context-learned model.
  arXiv  Detail & Related papers  (2025-02-08T00:16:44Z)
• On Fusing ChatGPT and Ensemble Learning in Discon-tinuous Named Entity Recognition in Health Corpora [0.0]
  We investigate the integration of ChatGPT as an arbitrator within an ensemble method, aiming to enhance performance on DNER tasks. Our method combines five state-of-the-art NER models with ChatGPT using custom prompt engineering to assess the robustness and generalization capabilities of the ensemble algorithm. The results indicate that our proposed fusion of ChatGPT with the ensemble learning algorithm outperforms the SOTA results in the CADEC, ShARe13, and ShARe14 datasets.
  arXiv  Detail & Related papers  (2024-12-22T11:26:49Z)
• How Well Do Text Embedding Models Understand Syntax? [50.440590035493074]
  The ability of text embedding models to generalize across a wide range of syntactic contexts remains under-explored. Our findings reveal that existing text embedding models have not sufficiently addressed these syntactic understanding challenges. We propose strategies to augment the generalization ability of text embedding models in diverse syntactic scenarios.
  arXiv  Detail & Related papers  (2023-11-14T08:51:00Z)
• Large Language Models Meet Open-World Intent Discovery and Recognition: An Evaluation of ChatGPT [37.27411474856601]
  Out-of-domain (OOD) intent discovery and generalized intent discovery (GID) aim to extend a closed intent to open-world intent sets. Previous methods address them by fine-tuning discriminative models. ChatGPT exhibits consistent advantages under zero-shot settings, but is still at a disadvantage compared to fine-tuned models.
  arXiv  Detail & Related papers  (2023-10-16T08:34:44Z)
• On the Generalization of Training-based ChatGPT Detection Methods [33.46128880100525]
  ChatGPT is one of the most popular language models which achieve amazing performance on various natural language tasks. There is also an urgent need to detect the texts generated ChatGPT from human written.
  arXiv  Detail & Related papers  (2023-10-02T16:13:08Z)
• HC3 Plus: A Semantic-Invariant Human ChatGPT Comparison Corpus [22.302137281411646]
  ChatGPT has garnered significant interest due to its impressive performance. There is growing concern about its potential risks. Current datasets used for detecting ChatGPT-generated text primarily focus on question-answering tasks.
  arXiv  Detail & Related papers  (2023-09-06T05:33:57Z)
• ChatGraph: Interpretable Text Classification by Converting ChatGPT Knowledge to Graphs [54.48467003509595]
  ChatGPT has shown superior performance in various natural language processing (NLP) tasks. We propose a novel framework that leverages the power of ChatGPT for specific tasks, such as text classification. Our method provides a more transparent decision-making process compared with previous text classification methods.
  arXiv  Detail & Related papers  (2023-05-03T19:57:43Z)
• To ChatGPT, or not to ChatGPT: That is the question! [78.407861566006]
  This study provides a comprehensive and contemporary assessment of the most recent techniques in ChatGPT detection. We have curated a benchmark dataset consisting of prompts from ChatGPT and humans, including diverse questions from medical, open Q&A, and finance domains. Our evaluation results demonstrate that none of the existing methods can effectively detect ChatGPT-generated content.
  arXiv  Detail & Related papers  (2023-04-04T03:04:28Z)
• A Human Word Association based model for topic detection in social networks [1.8749305679160366]
  This paper introduces a topic detection framework for social networks based on the concept of imitating the mental ability of word association. The performance of this framework is evaluated using the FA-CUP dataset, a benchmark in the field of topic detection.
  arXiv  Detail & Related papers  (2023-01-30T17:10:34Z)
• Novel Human-Object Interaction Detection via Adversarial Domain Generalization [103.55143362926388]
  We study the problem of novel human-object interaction (HOI) detection, aiming at improving the generalization ability of the model to unseen scenarios. The challenge mainly stems from the large compositional space of objects and predicates, which leads to the lack of sufficient training data for all the object-predicate combinations. We propose a unified framework of adversarial domain generalization to learn object-invariant features for predicate prediction.
  arXiv  Detail & Related papers  (2020-05-22T22:02:56Z)
• A Dependency Syntactic Knowledge Augmented Interactive Architecture for End-to-End Aspect-based Sentiment Analysis [73.74885246830611]
  We propose a novel dependency syntactic knowledge augmented interactive architecture with multi-task learning for end-to-end ABSA. This model is capable of fully exploiting the syntactic knowledge (dependency relations and types) by leveraging a well-designed Dependency Relation Embedded Graph Convolutional Network (DreGcn) Extensive experimental results on three benchmark datasets demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2020-04-04T14:59:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.