arXiVeri: Automatic table verification with GPT

• URL: http://arxiv.org/abs/2306.07968v1
• Date: Tue, 13 Jun 2023 17:59:57 GMT
• Title: arXiVeri: Automatic table verification with GPT
• Authors: Gyungin Shin, Weidi Xie, Samuel Albanie
• Abstract summary: We propose a novel task of automatic table verification (AutoTV) The objective is to verify the accuracy of numerical data in tables by cross-referencing cited sources. By leveraging the flexible capabilities of modern large language models (LLMs), we propose simple baselines for table verification.
• Score: 44.388120096898554
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Without accurate transcription of numerical data in scientific documents, a scientist cannot draw accurate conclusions. Unfortunately, the process of copying numerical data from one paper to another is prone to human error. In this paper, we propose to meet this challenge through the novel task of automatic table verification (AutoTV), in which the objective is to verify the accuracy of numerical data in tables by cross-referencing cited sources. To support this task, we propose a new benchmark, arXiVeri, which comprises tabular data drawn from open-access academic papers on arXiv. We introduce metrics to evaluate the performance of a table verifier in two key areas: (i) table matching, which aims to identify the source table in a cited document that corresponds to a target table, and (ii) cell matching, which aims to locate shared cells between a target and source table and identify their row and column indices accurately. By leveraging the flexible capabilities of modern large language models (LLMs), we propose simple baselines for table verification. Our findings highlight the complexity of this task, even for state-of-the-art LLMs like OpenAI's GPT-4. The code and benchmark will be made publicly available.

Related papers

• ArxivDIGESTables: Synthesizing Scientific Literature into Tables using Language Models [58.34560740973768]
  We introduce a framework that leverages language models (LMs) to generate literature review tables. A new dataset of 2,228 literature review tables extracted from ArXiv papers synthesize a total of 7,542 research papers. We evaluate LMs' abilities to reconstruct reference tables, finding this task benefits from additional context.
  arXiv  Detail & Related papers  (2024-10-25T18:31:50Z)
• TableLLM: Enabling Tabular Data Manipulation by LLMs in Real Office Usage Scenarios [52.73289223176475]
  TableLLM is a robust large language model (LLM) with 13 billion parameters. TableLLM is purpose-built for proficiently handling data manipulation tasks. We have released the model checkpoint, source code, benchmarks, and a web application for user interaction.
  arXiv  Detail & Related papers  (2024-03-28T11:21:12Z)
• TDeLTA: A Light-weight and Robust Table Detection Method based on Learning Text Arrangement [34.73880086005418]
  We propose a novel, light-weighted and robust Table Detection method based on Learning Text Arrangement, namely TDeLTA. To locate the tables precisely, we design a text-classification task, classifying the text blocks into 4 categories according to their semantic roles in the tables. Compared to several state-of-the-art methods, TDeLTA achieves competitive results with only 3.1M model parameters on the large-scale public datasets.
  arXiv  Detail & Related papers  (2023-12-18T09:18:43Z)
• Data augmentation on graphs for table type classification [1.1859913430860336]
  We address the classification of tables using a Graph Neural Network, exploiting the table structure for the message passing algorithm in use. We achieve promising preliminary results, proposing a data augmentation method suitable for graph-based table representation.
  arXiv  Detail & Related papers  (2022-08-23T21:54:46Z)
• Graph Neural Networks and Representation Embedding for Table Extraction in PDF Documents [1.1859913430860336]
  The main contribution of this work is to tackle the problem of table extraction, exploiting Graph Neural Networks. We experimentally evaluated the proposed approach on a new dataset obtained by merging the information provided in the PubLayNet and PubTables-1M datasets.
  arXiv  Detail & Related papers  (2022-08-23T21:36:01Z)
• TGRNet: A Table Graph Reconstruction Network for Table Structure Recognition [76.06530816349763]
  We propose an end-to-end trainable table graph reconstruction network (TGRNet) for table structure recognition. Specifically, the proposed method has two main branches, a cell detection branch and a cell logical location branch, to jointly predict the spatial location and the logical location of different cells.
  arXiv  Detail & Related papers  (2021-06-20T01:57:05Z)
• A Graph Representation of Semi-structured Data for Web Question Answering [96.46484690047491]
  We propose a novel graph representation of Web tables and lists based on a systematic categorization of the components in semi-structured data as well as their relations. Our method improves F1 score by 3.90 points over the state-of-the-art baselines.
  arXiv  Detail & Related papers  (2020-10-14T04:01:54Z)
• ToTTo: A Controlled Table-To-Text Generation Dataset [61.83159452483026]
  ToTTo is an open-domain English table-to-text dataset with over 120,000 training examples. We introduce a dataset construction process where annotators directly revise existing candidate sentences from Wikipedia. While usually fluent, existing methods often hallucinate phrases that are not supported by the table.
  arXiv  Detail & Related papers  (2020-04-29T17:53:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.