Consistency Training by Synthetic Question Generation for Conversational Question Answering

• URL: http://arxiv.org/abs/2404.11109v1
• Date: Wed, 17 Apr 2024 06:49:14 GMT
• Title: Consistency Training by Synthetic Question Generation for Conversational Question Answering
• Authors: Hamed Hematian Hemati, Hamid Beigy,
• Abstract summary: We augment historical information with synthetic questions to make the reasoning robust to irrelevant history. This is the first instance of research using question generation as a form of data augmentation to model conversational QA settings.
• Score: 14.211024633768986
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Efficiently modeling historical information is a critical component in addressing user queries within a conversational question-answering (QA) context, as historical context plays a vital role in clarifying the user's questions. However, irrelevant history induces noise in the reasoning process, especially for those questions with a considerable historical context. In our novel model-agnostic approach, referred to as CoTaH (Consistency-Trained augmented History), we augment the historical information with synthetic questions and subsequently employ consistency training to train a model that utilizes both real and augmented historical data to implicitly make the reasoning robust to irrelevant history. To the best of our knowledge, this is the first instance of research using question generation as a form of data augmentation to model conversational QA settings. By citing a common modeling error prevalent in previous research, we introduce a new baseline model and compare our model's performance against it, demonstrating an improvement in results, particularly when dealing with questions that include a substantial amount of historical context. The source code can be found on our GitHub page.

Related papers

• Long-Span Question-Answering: Automatic Question Generation and QA-System Ranking via Side-by-Side Evaluation [65.16137964758612]
  We explore the use of long-context capabilities in large language models to create synthetic reading comprehension data from entire books. Our objective is to test the capabilities of LLMs to analyze, understand, and reason over problems that require a detailed comprehension of long spans of text.
  arXiv  Detail & Related papers  (2024-05-31T20:15:10Z)
• Learning to Select the Relevant History Turns in Conversational Question Answering [27.049444003555234]
  The dependency between relevant history selection and correct answer prediction is an intriguing but under-explored area. We propose a framework, DHS-ConvQA, that first generates the context and question entities for all the history turns. We demonstrate that selecting relevant turns works better than rewriting the original question.
  arXiv  Detail & Related papers  (2023-08-04T12:59:39Z)
• Event Extraction as Question Generation and Answering [72.04433206754489]
  Recent work on Event Extraction has reframed the task as Question Answering (QA) We propose QGA-EE, which enables a Question Generation (QG) model to generate questions that incorporate rich contextual information instead of using fixed templates. Experiments show that QGA-EE outperforms all prior single-task-based models on the ACE05 English dataset.
  arXiv  Detail & Related papers  (2023-07-10T01:46:15Z)
• History Semantic Graph Enhanced Conversational KBQA with Temporal Information Modeling [28.27368343751272]
  We propose a History Semantic Graph Enhanced KBQA model (HSGE) that is able to effectively model long-range semantic dependencies in conversation history. We evaluate HSGE on a widely used benchmark dataset for complex sequential question answering.
  arXiv  Detail & Related papers  (2023-06-12T05:10:58Z)
• RECKONING: Reasoning through Dynamic Knowledge Encoding [51.076603338764706]
  We show that language models can answer questions by reasoning over knowledge provided as part of the context. In these situations, the model fails to distinguish the knowledge that is necessary to answer the question. We propose teaching the model to reason more robustly by folding the provided contextual knowledge into the model's parameters.
  arXiv  Detail & Related papers  (2023-05-10T17:54:51Z)
• Realistic Conversational Question Answering with Answer Selection based on Calibrated Confidence and Uncertainty Measurement [54.55643652781891]
  Conversational Question Answering (ConvQA) models aim at answering a question with its relevant paragraph and previous question-answer pairs that occurred during conversation multiple times. We propose to filter out inaccurate answers in the conversation history based on their estimated confidences and uncertainties from the ConvQA model. We validate our models, Answer Selection-based realistic Conversation Question Answering, on two standard ConvQA datasets.
  arXiv  Detail & Related papers  (2023-02-10T09:42:07Z)
• History-Aware Hierarchical Transformer for Multi-session Open-domain Dialogue System [59.78425104243993]
  We propose History-Aware Hierarchical Transformer (HAHT) for multi-session open-domain dialogue. HAHT maintains a long-term memory of history conversations and utilizes history information to understand current conversation context. Experimental results on a large-scale Multi-Session Conversation dataset suggest that the proposed HAHT model consistently outperforms baseline models.
  arXiv  Detail & Related papers  (2023-02-02T06:54:33Z)
• CoHS-CQG: Context and History Selection for Conversational Question Generation [31.87967788600221]
  We propose a two-stage CQG framework, which adopts a CoHS module to shorten the context and history of the input. Our model achieves state-of-the-art performances on CoQA in both the answer-aware and answer-unaware settings.
  arXiv  Detail & Related papers  (2022-09-14T13:58:52Z)
• BERT-CoQAC: BERT-based Conversational Question Answering in Context [10.811729691130349]
  We introduce a framework based on a publically available pre-trained language model called BERT for incorporating history turns into the system. Experiment results revealed that our framework is comparable in performance with the state-of-the-art models on the QuAC leader board.
  arXiv  Detail & Related papers  (2021-04-23T03:05:17Z)
• A Graph-guided Multi-round Retrieval Method for Conversational Open-domain Question Answering [52.041815783025186]
  We propose a novel graph-guided retrieval method to model the relations among answers across conversation turns. We also propose to incorporate the multi-round relevance feedback technique to explore the impact of the retrieval context on current question understanding.
  arXiv  Detail & Related papers  (2021-04-17T04:39:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.