How to Build Robust FAQ Chatbot with Controllable Question Generator?

• URL: http://arxiv.org/abs/2112.03007v1
• Date: Thu, 18 Nov 2021 12:54:07 GMT
• Title: How to Build Robust FAQ Chatbot with Controllable Question Generator?
• Authors: Yan Pan and Mingyang Ma and Bernhard Pflugfelder and Georg Groh
• Abstract summary: We propose a high-quality, diverse, controllable method to generate adversarial samples with a semantic graph. The fluent and semantically generated QA pairs fool our passage retrieval model successfully. We find that the generated data set improves the generalizability of the QA model to the new target domain.
• Score: 5.680871239968297
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Many unanswerable adversarial questions fool the question-answer (QA) system with some plausible answers. Building a robust, frequently asked questions (FAQ) chatbot needs a large amount of diverse adversarial examples. Recent question generation methods are ineffective at generating many high-quality and diverse adversarial question-answer pairs from unstructured text. We propose the diversity controllable semantically valid adversarial attacker (DCSA), a high-quality, diverse, controllable method to generate standard and adversarial samples with a semantic graph. The fluent and semantically generated QA pairs fool our passage retrieval model successfully. After that, we conduct a study on the robustness and generalization of the QA model with generated QA pairs among different domains. We find that the generated data set improves the generalizability of the QA model to the new target domain and the robustness of the QA model to detect unanswerable adversarial questions.

Related papers

• An Empirical Comparison of LM-based Question and Answer Generation Methods [79.31199020420827]
  Question and answer generation (QAG) consists of generating a set of question-answer pairs given a context. In this paper, we establish baselines with three different QAG methodologies that leverage sequence-to-sequence language model (LM) fine-tuning. Experiments show that an end-to-end QAG model, which is computationally light at both training and inference times, is generally robust and outperforms other more convoluted approaches.
  arXiv  Detail & Related papers  (2023-05-26T14:59:53Z)
• RoMQA: A Benchmark for Robust, Multi-evidence, Multi-answer Question Answering [87.18962441714976]
  We introduce RoMQA, the first benchmark for robust, multi-evidence, multi-answer question answering (QA) We evaluate state-of-the-art large language models in zero-shot, few-shot, and fine-tuning settings, and find that RoMQA is challenging. Our results show that RoMQA is a challenging benchmark for large language models, and provides a quantifiable test to build more robust QA methods.
  arXiv  Detail & Related papers  (2022-10-25T21:39:36Z)
• Improving Unsupervised Question Answering via Summarization-Informed Question Generation [47.96911338198302]
  Question Generation (QG) is the task of generating a plausible question for a passage, answer> pair. We make use of freely available news summary data, transforming declarative sentences into appropriate questions using dependency parsing, named entity recognition and semantic role labeling. The resulting questions are then combined with the original news articles to train an end-to-end neural QG model.
  arXiv  Detail & Related papers  (2021-09-16T13:08:43Z)
• GTM: A Generative Triple-Wise Model for Conversational Question Generation [36.33685095934868]
  We propose a generative triple-wise model with hierarchical variations for open-domain conversational question generation (CQG) Our method significantly improves the quality of questions in terms of fluency, coherence and diversity over competitive baselines.
  arXiv  Detail & Related papers  (2021-06-07T14:07:07Z)
• Counterfactual Variable Control for Robust and Interpretable Question Answering [57.25261576239862]
  Deep neural network based question answering (QA) models are neither robust nor explainable in many cases. In this paper, we inspect such spurious "capability" of QA models using causal inference. We propose a novel approach called Counterfactual Variable Control (CVC) that explicitly mitigates any shortcut correlation.
  arXiv  Detail & Related papers  (2020-10-12T10:09:05Z)
• Generating Diverse and Consistent QA pairs from Contexts with Information-Maximizing Hierarchical Conditional VAEs [62.71505254770827]
  We propose a conditional variational autoencoder (HCVAE) for generating QA pairs given unstructured texts as contexts. Our model obtains impressive performance gains over all baselines on both tasks, using only a fraction of data for training.
  arXiv  Detail & Related papers  (2020-05-28T08:26:06Z)
• Unsupervised Question Decomposition for Question Answering [102.56966847404287]
  We propose an algorithm for One-to-N Unsupervised Sequence Sequence (ONUS) that learns to map one hard, multi-hop question to many simpler, single-hop sub-questions. We show large QA improvements on HotpotQA over a strong baseline on the original, out-of-domain, and multi-hop dev sets.
  arXiv  Detail & Related papers  (2020-02-22T19:40:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.