SCAI-QReCC Shared Task on Conversational Question Answering

• URL: http://arxiv.org/abs/2201.11094v1
• Date: Wed, 26 Jan 2022 18:03:21 GMT
• Title: SCAI-QReCC Shared Task on Conversational Question Answering
• Authors: Svitlana Vakulenko, Johannes Kiesel, Maik Fr\"obe
• Abstract summary: SCAI'21 was organised as an independent on-line event and featured a shared task on conversational question answering. We identified evaluation of answer correctness in this settings as the major challenge and a current research gap. We conducted two crowdsourcing experiments to collect annotations for answer plausibility and faithfulness.
• Score: 7.428559907101379
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Search-Oriented Conversational AI (SCAI) is an established venue that regularly puts a spotlight upon the recent work advancing the field of conversational search. SCAI'21 was organised as an independent on-line event and featured a shared task on conversational question answering. Since all of the participant teams experimented with answer generation models for this task, we identified evaluation of answer correctness in this settings as the major challenge and a current research gap. Alongside the automatic evaluation, we conducted two crowdsourcing experiments to collect annotations for answer plausibility and faithfulness. As a result of this shared task, the original conversational QA dataset used for evaluation was further extended with alternative correct answers produced by the participant systems.

Related papers

• Boosting Conversational Question Answering with Fine-Grained Retrieval-Augmentation and Self-Check [25.63538452425097]
  We propose a conversation-level RAG approach, which incorporates fine-grained retrieval augmentation and self-check for conversational question answering. In particular, our approach consists of three components, namely conversational question refiner, fine-grained retriever and self-check based response generator.
  arXiv  Detail & Related papers  (2024-03-27T04:20:18Z)
• PAQA: Toward ProActive Open-Retrieval Question Answering [34.883834970415734]
  This work aims to tackle the challenge of generating relevant clarifying questions by taking into account the inherent ambiguities present in both user queries and documents. We propose PAQA, an extension to the existing AmbiNQ dataset, incorporating clarifying questions. We then evaluate various models and assess how passage retrieval impacts ambiguity detection and the generation of clarifying questions.
  arXiv  Detail & Related papers  (2024-02-26T14:40:34Z)
• Conversational QA Dataset Generation with Answer Revision [2.5838973036257458]
  We introduce a novel framework that extracts question-worthy phrases from a passage and then generates corresponding questions considering previous conversations. Our framework revises the extracted answers after generating questions so that answers exactly match paired questions.
  arXiv  Detail & Related papers  (2022-09-23T04:05:38Z)
• QAConv: Question Answering on Informative Conversations [85.2923607672282]
  We focus on informative conversations including business emails, panel discussions, and work channels. In total, we collect 34,204 QA pairs, including span-based, free-form, and unanswerable questions.
  arXiv  Detail & Related papers  (2021-05-14T15:53:05Z)
• Towards Data Distillation for End-to-end Spoken Conversational Question Answering [65.124088336738]
  We propose a new Spoken Conversational Question Answering task (SCQA) SCQA aims at enabling QA systems to model complex dialogues flow given the speech utterances and text corpora. Our main objective is to build a QA system to deal with conversational questions both in spoken and text forms.
  arXiv  Detail & Related papers  (2020-10-18T05:53:39Z)
• Response Selection for Multi-Party Conversations with Dynamic Topic Tracking [63.15158355071206]
  We frame response selection as a dynamic topic tracking task to match the topic between the response and relevant conversation context. We propose a novel multi-task learning framework that supports efficient encoding through large pretrained models. Experimental results on the DSTC-8 Ubuntu IRC dataset show state-of-the-art results in response selection and topic disentanglement tasks.
  arXiv  Detail & Related papers  (2020-10-15T14:21:38Z)
• ConvAI3: Generating Clarifying Questions for Open-Domain Dialogue Systems (ClariQ) [64.60303062063663]
  This document presents a detailed description of the challenge on clarifying questions for dialogue systems (ClariQ) The challenge is organized as part of the Conversational AI challenge series (ConvAI3) at Search Oriented Conversational AI (SCAI) EMNLP workshop in 2020.
  arXiv  Detail & Related papers  (2020-09-23T19:48:02Z)
• Learning an Effective Context-Response Matching Model with Self-Supervised Tasks for Retrieval-based Dialogues [88.73739515457116]
  We introduce four self-supervised tasks including next session prediction, utterance restoration, incoherence detection and consistency discrimination. We jointly train the PLM-based response selection model with these auxiliary tasks in a multi-task manner. Experiment results indicate that the proposed auxiliary self-supervised tasks bring significant improvement for multi-turn response selection.
  arXiv  Detail & Related papers  (2020-09-14T08:44:46Z)
• Open-Retrieval Conversational Question Answering [62.11228261293487]
  We introduce an open-retrieval conversational question answering (ORConvQA) setting, where we learn to retrieve evidence from a large collection before extracting answers. We build an end-to-end system for ORConvQA, featuring a retriever, a reranker, and a reader that are all based on Transformers.
  arXiv  Detail & Related papers  (2020-05-22T19:39:50Z)
• Do not let the history haunt you -- Mitigating Compounding Errors in Conversational Question Answering [17.36904526340775]
  We find that compounding errors occur when using previously predicted answers at test time. We propose a sampling strategy that dynamically selects between target answers and model predictions during training.
  arXiv  Detail & Related papers  (2020-05-12T13:29:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.