A Dataset of Open-Domain Question Answering with Multiple-Span Answers
- URL: http://arxiv.org/abs/2402.09923v1
- Date: Thu, 15 Feb 2024 13:03:57 GMT
- Title: A Dataset of Open-Domain Question Answering with Multiple-Span Answers
- Authors: Zhiyi Luo, Yingying Zhang, Shuyun Luo, Ying Zhao, Wentao Lyu
- Abstract summary: Multi-span answer extraction, also known as the task of multi-span question answering (MSQA), is critical for real-world applications.
There is a notable lack of publicly available MSQA benchmark in Chinese.
We present CLEAN, a comprehensive Chinese multi-span question answering dataset.
- Score: 11.291635421662338
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Multi-span answer extraction, also known as the task of multi-span question
answering (MSQA), is critical for real-world applications, as it requires
extracting multiple pieces of information from a text to answer complex
questions. Despite the active studies and rapid progress in English MSQA
research, there is a notable lack of publicly available MSQA benchmark in
Chinese. Previous efforts for constructing MSQA datasets predominantly
emphasized entity-centric contextualization, resulting in a bias towards
collecting factoid questions and potentially overlooking questions requiring
more detailed descriptive responses. To overcome these limitations, we present
CLEAN, a comprehensive Chinese multi-span question answering dataset that
involves a wide range of open-domain subjects with a substantial number of
instances requiring descriptive answers. Additionally, we provide established
models from relevant literature as baselines for CLEAN. Experimental results
and analysis show the characteristics and challenge of the newly proposed CLEAN
dataset for the community. Our dataset, CLEAN, will be publicly released at
zhiyiluo.site/misc/clean_v1.0_ sample.json.
Related papers
- Qsnail: A Questionnaire Dataset for Sequential Question Generation [76.616068047362]
We present the first dataset specifically constructed for the questionnaire generation task, which comprises 13,168 human-written questionnaires.
We conduct experiments on Qsnail, and the results reveal that retrieval models and traditional generative models do not fully align with the given research topic and intents.
Despite enhancements through the chain-of-thought prompt and finetuning, questionnaires generated by language models still fall short of human-written questionnaires.
arXiv Detail & Related papers (2024-02-22T04:14:10Z) - PCoQA: Persian Conversational Question Answering Dataset [12.07607688189035]
The PCoQA dataset is a resource comprising information-seeking dialogs encompassing a total of 9,026 contextually-driven questions.
PCoQA is designed to present novel challenges compared to previous question answering datasets.
This paper not only presents the comprehensive PCoQA dataset but also reports the performance of various benchmark models.
arXiv Detail & Related papers (2023-12-07T15:29:34Z) - SEMQA: Semi-Extractive Multi-Source Question Answering [94.04430035121136]
We introduce a new QA task for answering multi-answer questions by summarizing multiple diverse sources in a semi-extractive fashion.
We create the first dataset of this kind, QuoteSum, with human-written semi-extractive answers to natural and generated questions.
arXiv Detail & Related papers (2023-11-08T18:46:32Z) - UNK-VQA: A Dataset and a Probe into the Abstention Ability of Multi-modal Large Models [55.22048505787125]
This paper contributes a comprehensive dataset, called UNK-VQA.
We first augment the existing data via deliberate perturbations on either the image or question.
We then extensively evaluate the zero- and few-shot performance of several emerging multi-modal large models.
arXiv Detail & Related papers (2023-10-17T02:38:09Z) - Activity report analysis with automatic single or multispan answer
extraction [0.21485350418225244]
We create a new smart home environment dataset comprised of questions paired with single-span or multi-span answers depending on the question and context queried.
Our experiments show that the proposed model outperforms state-of-the-art QA models on our dataset.
arXiv Detail & Related papers (2022-09-09T06:33:29Z) - Towards Complex Document Understanding By Discrete Reasoning [77.91722463958743]
Document Visual Question Answering (VQA) aims to understand visually-rich documents to answer questions in natural language.
We introduce a new Document VQA dataset, named TAT-DQA, which consists of 3,067 document pages and 16,558 question-answer pairs.
We develop a novel model named MHST that takes into account the information in multi-modalities, including text, layout and visual image, to intelligently address different types of questions.
arXiv Detail & Related papers (2022-07-25T01:43:19Z) - Modern Question Answering Datasets and Benchmarks: A Survey [5.026863544662493]
Question Answering (QA) is one of the most important natural language processing (NLP) tasks.
It aims using NLP technologies to generate a corresponding answer to a given question based on the massive unstructured corpus.
In this paper, we investigate influential QA datasets that have been released in the era of deep learning.
arXiv Detail & Related papers (2022-06-30T05:53:56Z) - RxWhyQA: a clinical question-answering dataset with the challenge of
multi-answer questions [4.017119245460155]
We create a dataset for the development and evaluation of clinical question-answering systems that can handle multi-answer questions.
The 1-to-0 and 1-to-N drug-reason relations formed the unanswerable and multi-answer entries.
arXiv Detail & Related papers (2022-01-07T15:58:58Z) - AnswerSumm: A Manually-Curated Dataset and Pipeline for Answer
Summarization [73.91543616777064]
Community Question Answering (CQA) fora such as Stack Overflow and Yahoo! Answers contain a rich resource of answers to a wide range of community-based questions.
One goal of answer summarization is to produce a summary that reflects the range of answer perspectives.
This work introduces a novel dataset of 4,631 CQA threads for answer summarization, curated by professional linguists.
arXiv Detail & Related papers (2021-11-11T21:48:02Z) - Self-Teaching Machines to Read and Comprehend with Large-Scale
Multi-Subject Question Answering Data [58.36305373100518]
It is unclear whether subject-area question-answering data is useful for machine reading comprehension tasks.
We collect a large-scale multi-subject multiple-choice question-answering dataset, ExamQA.
We use incomplete and noisy snippets returned by a web search engine as the relevant context for each question-answering instance to convert it into a weakly-labeled MRC instance.
arXiv Detail & Related papers (2021-02-01T23:18:58Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.