HCT-QA: A Benchmark for Question Answering on Human-Centric Tables

• URL: http://arxiv.org/abs/2504.20047v1
• Date: Sun, 09 Mar 2025 11:02:11 GMT
• Title: HCT-QA: A Benchmark for Question Answering on Human-Centric Tables
• Authors: Mohammad S. Ahmad, Zan A. Naeem, Michaël Aupetit, Ahmed Elmagarmid, Mohamed Eltabakh, Xiasong Ma, Mourad Ouzzani, Chaoyi Ruan,
• Abstract summary: Human-centric tables (HCTs) possess a unique combination of high business value, intricate layouts, limited operational power at scale, and sometimes serve as the only data source for critical insights.<n>This paper describes HCT-QA, an extensive benchmark of HCTs, natural language queries, and related answers on thousands of tables.
• Score: 2.8913928436305505
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Tabular data embedded within PDF files, web pages, and other document formats are prevalent across numerous sectors such as government, engineering, science, and business. These human-centric tables (HCTs) possess a unique combination of high business value, intricate layouts, limited operational power at scale, and sometimes serve as the only data source for critical insights. However, their complexity poses significant challenges to traditional data extraction, processing, and querying methods. While current solutions focus on transforming these tables into relational formats for SQL queries, they fall short in handling the diverse and complex layouts of HCTs and hence being amenable to querying. This paper describes HCT-QA, an extensive benchmark of HCTs, natural language queries, and related answers on thousands of tables. Our dataset includes 2,188 real-world HCTs with 9,835 QA pairs and 4,679 synthetic tables with 67.5K QA pairs. While HCTs can be potentially processed by different type of query engines, in this paper, we focus on Large Language Models as potential engines and assess their ability in processing and querying such tables.

Related papers

• MMTU: A Massive Multi-Task Table Understanding and Reasoning Benchmark [70.47478110973042]
  We introduce MMTU, a large-scale benchmark with over 30K questions across 25 real-world table tasks.<n> MMTU is designed to comprehensively evaluate models ability to understand, reason, and manipulate real tables at the expert-level.<n>We show that MMTU require a combination of skills -- including table understanding, reasoning, and coding -- that remain challenging for today's frontier models.
  arXiv  Detail & Related papers  (2025-06-05T21:05:03Z)
• SynTQA: Synergistic Table-based Question Answering via Mixture of Text-to-SQL and E2E TQA [25.09488366689108]
  Text-to- parsing and end-to-end question answering (E2E TQA) are two main approaches for Table-based Question Answering task. Despite success on multiple benchmarks, they have yet to be compared and their synergy remains unexplored. We identify different strengths and weaknesses through evaluating state-of-the-art models on benchmark datasets.
  arXiv  Detail & Related papers  (2024-09-25T07:18:45Z)
• TabPedia: Towards Comprehensive Visual Table Understanding with Concept Synergy [81.76462101465354]
  We present a novel large vision-hugging model, TabPedia, equipped with a concept synergy mechanism. This unified framework allows TabPedia to seamlessly integrate VTU tasks, such as table detection, table structure recognition, table querying, and table question answering. To better evaluate the VTU task in real-world scenarios, we establish a new and comprehensive table VQA benchmark, ComTQA.
  arXiv  Detail & Related papers  (2024-06-03T13:54:05Z)
• KET-QA: A Dataset for Knowledge Enhanced Table Question Answering [63.56707527868466]
  We propose to use a knowledge base (KB) as the external knowledge source for TableQA. Every question requires the integration of information from both the table and the sub-graph to be answered. We design a retriever-reasoner structured pipeline model to extract pertinent information from the vast knowledge sub-graph.
  arXiv  Detail & Related papers  (2024-05-13T18:26:32Z)
• TANQ: An open domain dataset of table answered questions [15.323690523538572]
  TANQ is the first open domain question answering dataset where the answers require building tables from information across multiple sources.<n>We release the full source attribution for every cell in the resulting table and benchmark state-of-the-art language models in open, oracle, and closed book setups.<n>Our best-performing baseline, Gemini Flash reaches an overall F1 score of 60.7, lagging behind human performance by 12.3 points.
  arXiv  Detail & Related papers  (2024-05-13T14:07:20Z)
• Large Language Model for Table Processing: A Survey [18.32332372134988]
  This survey provides a comprehensive overview of table-related tasks. It covers traditional tasks like table question answering as well as emerging fields such as spreadsheet manipulation and table data analysis.
  arXiv  Detail & Related papers  (2024-02-04T00:47:53Z)
• Auto-Tables: Synthesizing Multi-Step Transformations to Relationalize Tables without Using Examples [24.208275772387683]
  Auto-Tables can automatically transform non-relational tables into standard relational forms for downstream analytics. Our evaluation suggests that Auto-Tables can successfully synthesize transformations for over 70% of test cases at interactive speeds.
  arXiv  Detail & Related papers  (2023-07-27T00:55:54Z)
• MultiTabQA: Generating Tabular Answers for Multi-Table Question Answering [61.48881995121938]
  Real-world queries are complex in nature, often over multiple tables in a relational database or web page. Our model, MultiTabQA, not only answers questions over multiple tables, but also generalizes to generate tabular answers.
  arXiv  Detail & Related papers  (2023-05-22T08:25:15Z)
• Towards Complex Document Understanding By Discrete Reasoning [77.91722463958743]
  Document Visual Question Answering (VQA) aims to understand visually-rich documents to answer questions in natural language. We introduce a new Document VQA dataset, named TAT-DQA, which consists of 3,067 document pages and 16,558 question-answer pairs. We develop a novel model named MHST that takes into account the information in multi-modalities, including text, layout and visual image, to intelligently address different types of questions.
  arXiv  Detail & Related papers  (2022-07-25T01:43:19Z)
• Table Retrieval May Not Necessitate Table-specific Model Design [83.27735758203089]
  We focus on the task of table retrieval, and ask: "is table-specific model design necessary for table retrieval?" Based on an analysis on a table-based portion of the Natural Questions dataset (NQ-table), we find that structure plays a negligible role in more than 70% of the cases. We then experiment with three modules to explicitly encode table structures, namely auxiliary row/column embeddings, hard attention masks, and soft relation-based attention biases. None of these yielded significant improvements, suggesting that table-specific model design may not be necessary for table retrieval.
  arXiv  Detail & Related papers  (2022-05-19T20:35:23Z)
• AIT-QA: Question Answering Dataset over Complex Tables in the Airline Industry [30.330772077451048]
  We introduce the domain-specific Table QA dataset AIT-QA (Industry Table QA) The dataset consists of 515 questions authored by human annotators on 116 tables extracted from public U.S. SEC filings. We also provide annotations pertaining to the nature of questions, marking those that require hierarchical headers, domain-specific terminology, and paraphrased forms.
  arXiv  Detail & Related papers  (2021-06-24T12:14:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.