Abstract: Neural natural language generation (NLG) and understanding (NLU) models are
data-hungry and require massive amounts of annotated data to be competitive.
Recent frameworks address this bottleneck with generative models that
synthesize weak labels at scale, where a small amount of training labels are
expert-curated and the rest of the data is automatically annotated. We follow
that approach, by automatically constructing a large-scale weakly-labeled data
with a fine-tuned GPT-2, and employ a semi-supervised framework to jointly
train the NLG and NLU models. The proposed framework adapts the parameter
updates to the models according to the estimated label-quality. On both the E2E
and Weather benchmarks, we show that this weakly supervised training paradigm
is an effective approach under low resource scenarios and outperforming
benchmark systems on both datasets when 100% of training data is used.