Dual Inference for Improving Language Understanding and Generation

• URL: http://arxiv.org/abs/2010.04246v2
• Date: Thu, 15 Oct 2020 02:10:48 GMT
• Title: Dual Inference for Improving Language Understanding and Generation
• Authors: Shang-Yu Su, Yung-Sung Chuang, Yun-Nung Chen
• Abstract summary: Natural language understanding (NLU) and Natural language generation (NLG) tasks hold a strong dual relationship. NLU aims at predicting semantic labels based on natural language utterances and NLG does the opposite. This paper proposes to leverage the duality in the inference stage without the need of retraining.
• Score: 35.251935231914366
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Natural language understanding (NLU) and Natural language generation (NLG) tasks hold a strong dual relationship, where NLU aims at predicting semantic labels based on natural language utterances and NLG does the opposite. The prior work mainly focused on exploiting the duality in model training in order to obtain the models with better performance. However, regarding the fast-growing scale of models in the current NLP area, sometimes we may have difficulty retraining whole NLU and NLG models. To better address the issue, this paper proposes to leverage the duality in the inference stage without the need of retraining. The experiments on three benchmark datasets demonstrate the effectiveness of the proposed method in both NLU and NLG, providing the great potential of practical usage.

Related papers

• A Survey of Knowledge Enhanced Pre-trained Language Models [78.56931125512295]
  We present a comprehensive review of Knowledge Enhanced Pre-trained Language Models (KE-PLMs) For NLU, we divide the types of knowledge into four categories: linguistic knowledge, text knowledge, knowledge graph (KG) and rule knowledge. The KE-PLMs for NLG are categorized into KG-based and retrieval-based methods.
  arXiv  Detail & Related papers  (2022-11-11T04:29:02Z)
• Near-Negative Distinction: Giving a Second Life to Human Evaluation Datasets [95.4182455942628]
  We propose Near-Negative Distinction (NND) that repurposes prior human annotations into NND tests. In an NND test, an NLG model must place higher likelihood on a high-quality output candidate than on a near-negative candidate with a known error. We show that NND achieves higher correlation with human judgments than standard NLG evaluation metrics.
  arXiv  Detail & Related papers  (2022-05-13T20:02:53Z)
• A Survey of Knowledge-Intensive NLP with Pre-Trained Language Models [185.08295787309544]
  We aim to summarize the current progress of pre-trained language model-based knowledge-enhanced models (PLMKEs) We present the challenges of PLMKEs based on the discussion regarding the three elements and attempt to provide NLP practitioners with potential directions for further research.
  arXiv  Detail & Related papers  (2022-02-17T17:17:43Z)
• AdaPrompt: Adaptive Model Training for Prompt-based NLP [77.12071707955889]
  We propose AdaPrompt, adaptively retrieving external data for continual pretraining of PLMs. Experimental results on five NLP benchmarks show that AdaPrompt can improve over standard PLMs in few-shot settings. In zero-shot settings, our method outperforms standard prompt-based methods by up to 26.35% relative error reduction.
  arXiv  Detail & Related papers  (2022-02-10T04:04:57Z)
• Towards More Robust Natural Language Understanding [0.0]
  Natural Language Understanding (NLU) is branch of Natural Language Processing (NLP) Recent years have witnessed notable progress across various NLU tasks with deep learning techniques. It's worth noting that the human ability of understanding natural language is flexible and robust.
  arXiv  Detail & Related papers  (2021-12-01T17:27:19Z)
• A Generative Model for Joint Natural Language Understanding and Generation [9.810053382574017]
  We propose a generative model which couples NLU and NLG through a shared latent variable. Our model achieves state-of-the-art performance on two dialogue datasets with both flat and tree-structured formal representations. We also show that the model can be trained in a semi-supervised fashion by utilising unlabelled data to boost its performance.
  arXiv  Detail & Related papers  (2020-06-12T22:38:55Z)
• Towards Unsupervised Language Understanding and Generation by Joint Dual Learning [40.730699588561805]
  In modular dialogue systems, natural language understanding (NLU) and natural language generation (NLG) are critical components. This paper introduces a general learning framework to effectively exploit such duality. The proposed approach is capable of boosting the performance of both NLU and NLG.
  arXiv  Detail & Related papers  (2020-04-30T12:02:33Z)
• Boosting Naturalness of Language in Task-oriented Dialogues via Adversarial Training [29.468502787886813]
  We propose to integrate adversarial training to produce more human-like responses. In the RNN-LG Restaurant dataset, our model AdvNLG outperforms the previous state-of-the-art result by 3.6% in BLEU.
  arXiv  Detail & Related papers  (2020-04-30T03:35:20Z)
• Dual Learning for Semi-Supervised Natural Language Understanding [29.692288627633374]
  Natural language understanding (NLU) converts sentences into structured semantic forms. We introduce a dual task of NLU, semantic-to-sentence generation (SSG) We propose a new framework for semi-supervised NLU with the corresponding dual model.
  arXiv  Detail & Related papers  (2020-04-26T07:17:48Z)
• Logical Natural Language Generation from Open-Domain Tables [107.04385677577862]
  We propose a new task where a model is tasked with generating natural language statements that can be emphlogically entailed by the facts. To facilitate the study of the proposed logical NLG problem, we use the existing TabFact dataset citechen 2019tabfact featured with a wide range of logical/symbolic inferences. The new task poses challenges to the existing monotonic generation frameworks due to the mismatch between sequence order and logical order.
  arXiv  Detail & Related papers  (2020-04-22T06:03:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.