HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Data

• URL: http://arxiv.org/abs/2004.07347v3
• Date: Tue, 11 May 2021 23:29:14 GMT
• Title: HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Data
• Authors: Wenhu Chen, Hanwen Zha, Zhiyu Chen, Wenhan Xiong, Hong Wang, William Wang
• Abstract summary: We present HybridQA, a new large-scale question-answering dataset that requires reasoning on heterogeneous information. Each question is aligned with a Wikipedia table and multiple free-form corpora linked with the entities in the table. Tests show that the EM scores obtained by two baselines are below 20%, while the hybrid model can achieve an EM over 40%.
• Score: 39.91331662575689
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing question answering datasets focus on dealing with homogeneous information, based either only on text or KB/Table information alone. However, as human knowledge is distributed over heterogeneous forms, using homogeneous information alone might lead to severe coverage problems. To fill in the gap, we present HybridQA https://github.com/wenhuchen/HybridQA, a new large-scale question-answering dataset that requires reasoning on heterogeneous information. Each question is aligned with a Wikipedia table and multiple free-form corpora linked with the entities in the table. The questions are designed to aggregate both tabular information and text information, i.e., lack of either form would render the question unanswerable. We test with three different models: 1) a table-only model. 2) text-only model. 3) a hybrid model that combines heterogeneous information to find the answer. The experimental results show that the EM scores obtained by two baselines are below 20\%, while the hybrid model can achieve an EM over 40\%. This gap suggests the necessity to aggregate heterogeneous information in HybridQA. However, the hybrid model's score is still far behind human performance. Hence, HybridQA can serve as a challenging benchmark to study question answering with heterogeneous information.

Related papers

• Hybrid-SQuAD: Hybrid Scholarly Question Answering Dataset [8.867885891794877]
  We introduce Hybrid-SQuAD, a novel large-scale Scholarly Question Answering dataset. The dataset consists of 10.5K question-answer pairs generated by a large language model. We propose a RAG-based baseline hybrid QA model, achieving an exact match score of 69.65 on the Hybrid-SQuAD test set.
  arXiv  Detail & Related papers  (2024-12-03T19:37:00Z)
• PCoQA: Persian Conversational Question Answering Dataset [12.07607688189035]
  The PCoQA dataset is a resource comprising information-seeking dialogs encompassing a total of 9,026 contextually-driven questions. PCoQA is designed to present novel challenges compared to previous question answering datasets. This paper not only presents the comprehensive PCoQA dataset but also reports the performance of various benchmark models.
  arXiv  Detail & Related papers  (2023-12-07T15:29:34Z)
• A Lightweight Method to Generate Unanswerable Questions in English [18.323248259867356]
  We examine a simpler data augmentation method for unanswerable question generation in English. We perform antonym and entity swaps on answerable questions. Compared to the prior state-of-the-art, data generated with our training-free and lightweight strategy results in better models.
  arXiv  Detail & Related papers  (2023-10-30T10:14:52Z)
• UNK-VQA: A Dataset and a Probe into the Abstention Ability of Multi-modal Large Models [55.22048505787125]
  This paper contributes a comprehensive dataset, called UNK-VQA. We first augment the existing data via deliberate perturbations on either the image or question. We then extensively evaluate the zero- and few-shot performance of several emerging multi-modal large models.
  arXiv  Detail & Related papers  (2023-10-17T02:38:09Z)
• TACR: A Table-alignment-based Cell-selection and Reasoning Model for Hybrid Question-Answering [31.79113994947629]
  We propose a Table-alignment-based Cell-selection and Reasoning model (TACR) for hybrid text and table QA. In evidence retrieval, we design a table-question-alignment enhanced cell-selection method to retrieve fine-grained evidence. In answer reasoning, we incorporate a QA module that treats the row containing selected cells as context.
  arXiv  Detail & Related papers  (2023-05-24T03:42:44Z)
• HeteroQA: Learning towards Question-and-Answering through Multiple Information Sources via Heterogeneous Graph Modeling [50.39787601462344]
  Community Question Answering (CQA) is a well-defined task that can be used in many scenarios, such as E-Commerce and online user community for special interests. Most of the CQA methods only incorporate articles or Wikipedia to extract knowledge and answer the user's question. We propose a question-aware heterogeneous graph transformer to incorporate the multiple information sources (MIS) in the user community to automatically generate the answer.
  arXiv  Detail & Related papers  (2021-12-27T10:16:43Z)
• MixQG: Neural Question Generation with Mixed Answer Types [54.23205265351248]
  We propose a neural question generator, MixQG, to bridge this gap. We combine 9 question answering datasets with diverse answer types, including yes/no, multiple-choice, extractive, and abstractive answers. Our model outperforms existing work in both seen and unseen domains.
  arXiv  Detail & Related papers  (2021-10-15T16:03:40Z)
• Dual Reader-Parser on Hybrid Textual and Tabular Evidence for Open Domain Question Answering [78.9863753810787]
  A large amount of world's knowledge is stored in structured databases. query languages can answer questions that require complex reasoning, as well as offering full explainability.
  arXiv  Detail & Related papers  (2021-08-05T22:04:13Z)
• TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance [71.76018597965378]
  We build a new large-scale Question Answering dataset containing both Tabular And Textual data, named TAT-QA. We propose a novel QA model termed TAGOP, which is capable of reasoning over both tables and text.
  arXiv  Detail & Related papers  (2021-05-17T06:12:06Z)
• FeTaQA: Free-form Table Question Answering [33.018256483762386]
  We introduce FeTaQA, a new dataset with 10K Wikipedia-based table, question, free-form answer, supporting table cells pairs. FeTaQA yields a more challenging table question answering setting because it requires generating free-form text answers after retrieval, inference, and integration of multiple discontinuous facts from a structured knowledge source.
  arXiv  Detail & Related papers  (2021-04-01T09:59:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.