A Single Example Can Improve Zero-Shot Data Generation
- URL: http://arxiv.org/abs/2108.06991v1
- Date: Mon, 16 Aug 2021 09:43:26 GMT
- Title: A Single Example Can Improve Zero-Shot Data Generation
- Authors: Pavel Burnyshev, Valentin Malykh, Andrey Bout, Ekaterina Artemova,
Irina Piontkovskaya
- Abstract summary: Sub-tasks of intent classification require extensive and flexible datasets for experiments and evaluation.
We propose to use text generation methods to gather datasets.
We explore two approaches to generating task-oriented utterances.
- Score: 7.237231992155901
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Sub-tasks of intent classification, such as robustness to distribution shift,
adaptation to specific user groups and personalization, out-of-domain
detection, require extensive and flexible datasets for experiments and
evaluation. As collecting such datasets is time- and labor-consuming, we
propose to use text generation methods to gather datasets. The generator should
be trained to generate utterances that belong to the given intent. We explore
two approaches to generating task-oriented utterances. In the zero-shot
approach, the model is trained to generate utterances from seen intents and is
further used to generate utterances for intents unseen during training. In the
one-shot approach, the model is presented with a single utterance from a test
intent. We perform a thorough automatic, and human evaluation of the dataset
generated utilizing two proposed approaches. Our results reveal that the
attributes of the generated data are close to original test sets, collected via
crowd-sourcing.
Related papers
- Personalized Federated Learning via Active Sampling [50.456464838807115]
This paper proposes a novel method for sequentially identifying similar (or relevant) data generators.
Our method evaluates the relevance of a data generator by evaluating the effect of a gradient step using its local dataset.
We extend this method to non-parametric models by a suitable generalization of the gradient step to update a hypothesis using the local dataset provided by a data generator.
arXiv Detail & Related papers (2024-09-03T17:12:21Z) - Generating Survival Interpretable Trajectories and Data [2.4861619769660637]
The paper demonstrates the efficiency and properties of the proposed model using numerical experiments on synthetic and real datasets.
The code of the algorithm implementing the proposed model is publicly available.
arXiv Detail & Related papers (2024-02-19T18:02:10Z) - Dual Student Networks for Data-Free Model Stealing [79.67498803845059]
Two main challenges are estimating gradients of the target model without access to its parameters, and generating a diverse set of training samples.
We propose a Dual Student method where two students are symmetrically trained in order to provide the generator a criterion to generate samples that the two students disagree on.
We show that our new optimization framework provides more accurate gradient estimation of the target model and better accuracies on benchmark classification datasets.
arXiv Detail & Related papers (2023-09-18T18:11:31Z) - Self-augmented Data Selection for Few-shot Dialogue Generation [18.794770678708637]
We adopt the self-training framework to deal with the few-shot MR-to-Text generation problem.
We propose a novel data selection strategy to select the data that our generation model is most uncertain about.
arXiv Detail & Related papers (2022-05-19T16:25:50Z) - A Generative Language Model for Few-shot Aspect-Based Sentiment Analysis [90.24921443175514]
We focus on aspect-based sentiment analysis, which involves extracting aspect term, category, and predicting their corresponding polarities.
We propose to reformulate the extraction and prediction tasks into the sequence generation task, using a generative language model with unidirectional attention.
Our approach outperforms the previous state-of-the-art (based on BERT) on average performance by a large margins in few-shot and full-shot settings.
arXiv Detail & Related papers (2022-04-11T18:31:53Z) - Generative Conversational Networks [67.13144697969501]
We propose a framework called Generative Conversational Networks, in which conversational agents learn to generate their own labelled training data.
We show an average improvement of 35% in intent detection and 21% in slot tagging over a baseline model trained from the seed data.
arXiv Detail & Related papers (2021-06-15T23:19:37Z) - One-shot Learning with Absolute Generalization [23.77607345586489]
We propose a set of definitions to explain what kind of datasets can support one-shot learning.
Based on these definitions, we proposed a method to build an absolutely generalizable classifier.
Experiments demonstrate that the proposed method is superior to baseline on one-shot learning datasets and artificial datasets.
arXiv Detail & Related papers (2021-05-28T02:52:52Z) - Discriminative, Generative and Self-Supervised Approaches for
Target-Agnostic Learning [8.666667951130892]
generative and self-supervised learning models are shown to perform well at the task.
Our derived theorem for the pseudo-likelihood theory also shows that they are related for inferring a joint distribution model.
arXiv Detail & Related papers (2020-11-12T15:03:40Z) - Unsupervised Paraphrasing with Pretrained Language Models [85.03373221588707]
We propose a training pipeline that enables pre-trained language models to generate high-quality paraphrases in an unsupervised setting.
Our recipe consists of task-adaptation, self-supervision, and a novel decoding algorithm named Dynamic Blocking.
We show with automatic and human evaluations that our approach achieves state-of-the-art performance on both the Quora Question Pair and the ParaNMT datasets.
arXiv Detail & Related papers (2020-10-24T11:55:28Z) - Summary-Source Proposition-level Alignment: Task, Datasets and
Supervised Baseline [94.0601799665342]
Aligning sentences in a reference summary with their counterparts in source documents was shown as a useful auxiliary summarization task.
We propose establishing summary-source alignment as an explicit task, while introducing two major novelties.
We create a novel training dataset for proposition-level alignment, derived automatically from available summarization evaluation data.
We present a supervised proposition alignment baseline model, showing improved alignment-quality over the unsupervised approach.
arXiv Detail & Related papers (2020-09-01T17:27:12Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.