ConditionalQA: A Complex Reading Comprehension Dataset with Conditional Answers

• URL: http://arxiv.org/abs/2110.06884v1
• Date: Wed, 13 Oct 2021 17:16:46 GMT
• Title: ConditionalQA: A Complex Reading Comprehension Dataset with Conditional Answers
• Authors: Haitian Sun, William W. Cohen, Ruslan Salakhutdinov
• Abstract summary: We describe a Question Answering dataset that contains complex questions with conditional answers. We call this dataset ConditionalQA. We show that ConditionalQA is challenging for many of the existing QA models, especially in selecting answer conditions.
• Score: 93.55268936974971
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We describe a Question Answering (QA) dataset that contains complex questions with conditional answers, i.e. the answers are only applicable when certain conditions apply. We call this dataset ConditionalQA. In addition to conditional answers, the dataset also features: (1) long context documents with information that is related in logically complex ways; (2) multi-hop questions that require compositional logical reasoning; (3) a combination of extractive questions, yes/no questions, questions with multiple answers, and not-answerable questions; (4) questions asked without knowing the answers. We show that ConditionalQA is challenging for many of the existing QA models, especially in selecting answer conditions. We believe that this dataset will motivate further research in answering complex questions over long documents. Data and leaderboard are publicly available at \url{https://github.com/haitian-sun/ConditionalQA}.

Related papers

• Long-form Question Answering: An Iterative Planning-Retrieval-Generation Approach [28.849548176802262]
  Long-form question answering (LFQA) poses a challenge as it involves generating detailed answers in the form of paragraphs. We propose an LFQA model with iterative Planning, Retrieval, and Generation. We find that our model outperforms the state-of-the-art models on various textual and factual metrics for the LFQA task.
  arXiv  Detail & Related papers  (2023-11-15T21:22:27Z)
• Answering Ambiguous Questions with a Database of Questions, Answers, and Revisions [95.92276099234344]
  We present a new state-of-the-art for answering ambiguous questions that exploits a database of unambiguous questions generated from Wikipedia. Our method improves performance by 15% on recall measures and 10% on measures which evaluate disambiguating questions from predicted outputs.
  arXiv  Detail & Related papers  (2023-08-16T20:23:16Z)
• Reasoning over Logically Interacted Conditions for Question Answering [113.9231035680578]
  We study a more challenging task where answers are constrained by a list of conditions that logically interact. We propose a new model, TReasoner, for this challenging reasoning task. TReasoner achieves state-of-the-art performance on two benchmark conditional QA datasets.
  arXiv  Detail & Related papers  (2022-05-25T16:41:39Z)
• SituatedQA: Incorporating Extra-Linguistic Contexts into QA [7.495151447459443]
  We introduce SituatedQA, an open-retrieval QA dataset where systems must produce the correct answer to a question given the temporal or geographical context. We find that a significant proportion of information seeking questions have context-dependent answers. Our study shows that existing models struggle with producing answers that are frequently updated or from uncommon locations.
  arXiv  Detail & Related papers  (2021-09-13T17:53:21Z)
• QAConv: Question Answering on Informative Conversations [85.2923607672282]
  We focus on informative conversations including business emails, panel discussions, and work channels. In total, we collect 34,204 QA pairs, including span-based, free-form, and unanswerable questions.
  arXiv  Detail & Related papers  (2021-05-14T15:53:05Z)
• A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers [66.11048565324468]
  We present a dataset of 5,049 questions over 1,585 Natural Language Processing papers. Each question is written by an NLP practitioner who read only the title and abstract of the corresponding paper, and the question seeks information present in the full text. We find that existing models that do well on other QA tasks do not perform well on answering these questions, underperforming humans by at least 27 F1 points when answering them from entire papers.
  arXiv  Detail & Related papers  (2021-05-07T00:12:34Z)
• GooAQ: Open Question Answering with Diverse Answer Types [63.06454855313667]
  We present GooAQ, a large-scale dataset with a variety of answer types. This dataset contains over 5 million questions and 3 million answers collected from Google.
  arXiv  Detail & Related papers  (2021-04-18T05:40:39Z)
• Challenges in Information-Seeking QA: Unanswerable Questions and Paragraph Retrieval [46.3246135936476]
  We analyze why answering information-seeking queries is more challenging and where their prevalent unanswerabilities arise. Our controlled experiments suggest two headrooms -- paragraph selection and answerability prediction. We manually annotate 800 unanswerable examples across six languages on what makes them challenging to answer.
  arXiv  Detail & Related papers  (2020-10-22T17:48:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.