A Generative Model for Joint Natural Language Understanding and Generation

• URL: http://arxiv.org/abs/2006.07499v1
• Date: Fri, 12 Jun 2020 22:38:55 GMT
• Title: A Generative Model for Joint Natural Language Understanding and Generation
• Authors: Bo-Hsiang Tseng, Jianpeng Cheng, Yimai Fang, David Vandyke
• Abstract summary: We propose a generative model which couples NLU and NLG through a shared latent variable. Our model achieves state-of-the-art performance on two dialogue datasets with both flat and tree-structured formal representations. We also show that the model can be trained in a semi-supervised fashion by utilising unlabelled data to boost its performance.
• Score: 9.810053382574017
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Natural language understanding (NLU) and natural language generation (NLG) are two fundamental and related tasks in building task-oriented dialogue systems with opposite objectives: NLU tackles the transformation from natural language to formal representations, whereas NLG does the reverse. A key to success in either task is parallel training data which is expensive to obtain at a large scale. In this work, we propose a generative model which couples NLU and NLG through a shared latent variable. This approach allows us to explore both spaces of natural language and formal representations, and facilitates information sharing through the latent space to eventually benefit NLU and NLG. Our model achieves state-of-the-art performance on two dialogue datasets with both flat and tree-structured formal representations. We also show that the model can be trained in a semi-supervised fashion by utilising unlabelled data to boost its performance.

Related papers

• Natural Language Processing for Dialects of a Language: A Survey [56.93337350526933]
  State-of-the-art natural language processing (NLP) models are trained on massive training corpora, and report a superlative performance on evaluation datasets. This survey delves into an important attribute of these datasets: the dialect of a language. Motivated by the performance degradation of NLP models for dialectic datasets and its implications for the equity of language technologies, we survey past research in NLP for dialects in terms of datasets, and approaches.
  arXiv  Detail & Related papers  (2024-01-11T03:04:38Z)
• Large Language Models with Controllable Working Memory [64.71038763708161]
  Large language models (LLMs) have led to a series of breakthroughs in natural language processing (NLP) What further sets these models apart is the massive amounts of world knowledge they internalize during pretraining. How the model's world knowledge interacts with the factual information presented in the context remains under explored.
  arXiv  Detail & Related papers  (2022-11-09T18:58:29Z)
• SLM: Learning a Discourse Language Representation with Sentence Unshuffling [53.42814722621715]
  We introduce Sentence-level Language Modeling, a new pre-training objective for learning a discourse language representation. We show that this feature of our model improves the performance of the original BERT by large margins.
  arXiv  Detail & Related papers  (2020-10-30T13:33:41Z)
• Unsupervised Paraphrasing with Pretrained Language Models [85.03373221588707]
  We propose a training pipeline that enables pre-trained language models to generate high-quality paraphrases in an unsupervised setting. Our recipe consists of task-adaptation, self-supervision, and a novel decoding algorithm named Dynamic Blocking. We show with automatic and human evaluations that our approach achieves state-of-the-art performance on both the Quora Question Pair and the ParaNMT datasets.
  arXiv  Detail & Related papers  (2020-10-24T11:55:28Z)
• Dual Inference for Improving Language Understanding and Generation [35.251935231914366]
  Natural language understanding (NLU) and Natural language generation (NLG) tasks hold a strong dual relationship. NLU aims at predicting semantic labels based on natural language utterances and NLG does the opposite. This paper proposes to leverage the duality in the inference stage without the need of retraining.
  arXiv  Detail & Related papers  (2020-10-08T20:14:41Z)
• Language Models as Few-Shot Learner for Task-Oriented Dialogue Systems [74.8759568242933]
  Task-oriented dialogue systems use four connected modules, namely, Natural Language Understanding (NLU), a Dialogue State Tracking (DST), Dialogue Policy (DP) and Natural Language Generation (NLG) A research challenge is to learn each module with the least amount of samples given the high cost related to the data collection. We evaluate the priming few-shot ability of language models in the NLU, DP and NLG tasks.
  arXiv  Detail & Related papers  (2020-08-14T08:23:21Z)
• Schema-Guided Natural Language Generation [13.11874946084068]
  We present the novel task ofGuided Natural Language Generation (SG-NLG) In SG-NLG, the goal is still to generate a natural language prompt, but in SG-NLG, the input MRs are paired with rich schemata providing contextual information. We train different state-of-the-art models for neural natural language generation on this dataset and show that in many cases, including rich schema information allows our models to produce higher quality outputs.
  arXiv  Detail & Related papers  (2020-05-11T23:01:22Z)
• Towards Unsupervised Language Understanding and Generation by Joint Dual Learning [40.730699588561805]
  In modular dialogue systems, natural language understanding (NLU) and natural language generation (NLG) are critical components. This paper introduces a general learning framework to effectively exploit such duality. The proposed approach is capable of boosting the performance of both NLU and NLG.
  arXiv  Detail & Related papers  (2020-04-30T12:02:33Z)
• Boosting Naturalness of Language in Task-oriented Dialogues via Adversarial Training [29.468502787886813]
  We propose to integrate adversarial training to produce more human-like responses. In the RNN-LG Restaurant dataset, our model AdvNLG outperforms the previous state-of-the-art result by 3.6% in BLEU.
  arXiv  Detail & Related papers  (2020-04-30T03:35:20Z)
• Dual Learning for Semi-Supervised Natural Language Understanding [29.692288627633374]
  Natural language understanding (NLU) converts sentences into structured semantic forms. We introduce a dual task of NLU, semantic-to-sentence generation (SSG) We propose a new framework for semi-supervised NLU with the corresponding dual model.
  arXiv  Detail & Related papers  (2020-04-26T07:17:48Z)
• Logical Natural Language Generation from Open-Domain Tables [107.04385677577862]
  We propose a new task where a model is tasked with generating natural language statements that can be emphlogically entailed by the facts. To facilitate the study of the proposed logical NLG problem, we use the existing TabFact dataset citechen 2019tabfact featured with a wide range of logical/symbolic inferences. The new task poses challenges to the existing monotonic generation frameworks due to the mismatch between sequence order and logical order.
  arXiv  Detail & Related papers  (2020-04-22T06:03:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.