UniTable: Towards a Unified Framework for Table Recognition via Self-Supervised Pretraining

• URL: http://arxiv.org/abs/2403.04822v2
• Date: Mon, 27 May 2024 15:39:51 GMT
• Title: UniTable: Towards a Unified Framework for Table Recognition via Self-Supervised Pretraining
• Authors: ShengYun Peng, Aishwarya Chakravarthy, Seongmin Lee, Xiaojing Wang, Rajarajeswari Balasubramaniyan, Duen Horng Chau,
• Abstract summary: We present UniTable, a training framework that unifies the training paradigm and training objective of table recognition. Our framework unifies the training objectives of all three TR tasks into a unified task-agnostic training objective: language modeling. UniTable's table parsing capability has surpassed both existing TR methods and general large vision-language models.
• Score: 22.031699293366486
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Tables convey factual and quantitative data with implicit conventions created by humans that are often challenging for machines to parse. Prior work on table recognition (TR) has mainly centered around complex task-specific combinations of available inputs and tools. We present UniTable, a training framework that unifies both the training paradigm and training objective of TR. Its training paradigm combines the simplicity of purely pixel-level inputs with the effectiveness and scalability empowered by self-supervised pretraining from diverse unannotated tabular images. Our framework unifies the training objectives of all three TR tasks - extracting table structure, cell content, and cell bounding box - into a unified task-agnostic training objective: language modeling. Extensive quantitative and qualitative analyses highlight UniTable's state-of-the-art (SOTA) performance on four of the largest TR datasets. UniTable's table parsing capability has surpassed both existing TR methods and general large vision-language models, e.g., GPT-4o, GPT-4-turbo with vision, and LLaVA. Our code is publicly available at https://github.com/poloclub/unitable, featuring a Jupyter Notebook that includes the complete inference pipeline, fine-tuned across multiple TR datasets, supporting all three TR tasks.

Related papers

• Table-LLM-Specialist: Language Model Specialists for Tables using Iterative Generator-Validator Fine-tuning [52.08794743921141]
  We propose Table-LLM-Specialist, or Table-Specialist for short, as a new self-trained fine-tuning paradigm specifically designed for table tasks.
  arXiv  Detail & Related papers  (2024-10-16T02:04:17Z)
• TART: An Open-Source Tool-Augmented Framework for Explainable Table-based Reasoning [61.14586098005874]
  Current Large Language Models (LLMs) exhibit limited ability to understand table structures and to apply precise numerical reasoning. We introduce our Tool-Augmented Reasoning framework for Tables (TART), which integrates LLMs with specialized tools. TART contains three key components: a table formatter to ensure accurate data representation, a tool maker to develop specific computational tools, and an explanation generator to maintain explainability.
  arXiv  Detail & Related papers  (2024-09-18T06:19:59Z)
• TabPedia: Towards Comprehensive Visual Table Understanding with Concept Synergy [81.76462101465354]
  We present a novel large vision-hugging model, TabPedia, equipped with a concept synergy mechanism. This unified framework allows TabPedia to seamlessly integrate VTU tasks, such as table detection, table structure recognition, table querying, and table question answering. To better evaluate the VTU task in real-world scenarios, we establish a new and comprehensive table VQA benchmark, ComTQA.
  arXiv  Detail & Related papers  (2024-06-03T13:54:05Z)
• Large Language Models are Complex Table Parsers [26.66460264175336]
  We propose to incorporate GPT-3.5 to address the challenges posed by Complex Table QA. Specifically, we encode each cell's hierarchical structure, position information and content as datasets. By enhancing the prompt template with an explanatory description of the meaning of each task, we effectively improve the hierarchical awareness structure capability.
  arXiv  Detail & Related papers  (2023-12-13T01:34:42Z)
• A large-scale dataset for end-to-end table recognition in the wild [13.717478398235055]
  Table recognition (TR) is one of the research hotspots in pattern recognition. Currently, the end-to-end TR in real scenarios, accomplishing the three sub-tasks simultaneously, is yet an unexplored research area. We propose a new large-scale dataset named Table Recognition Set (TabRecSet) with diverse table forms sourcing from multiple scenarios in the wild.
  arXiv  Detail & Related papers  (2023-03-27T02:48:51Z)
• Improving Cross-task Generalization of Unified Table-to-text Models with Compositional Task Configurations [63.04466647849211]
  Methods typically encode task information with a simple dataset name as a prefix to the encoder. We propose compositional task configurations, a set of prompts prepended to the encoder to improve cross-task generalization. We show this not only allows the model to better learn shared knowledge across different tasks at training, but also allows us to control the model by composing new configurations.
  arXiv  Detail & Related papers  (2022-12-17T02:20:14Z)
• SubTab: Subsetting Features of Tabular Data for Self-Supervised Representation Learning [5.5616364225463055]
  We introduce a new framework, Subsetting features of Tabular data (SubTab) In this paper, we introduce a new framework, Subsetting features of Tabular data (SubTab) We argue that reconstructing the data from the subset of its features rather than its corrupted version in an autoencoder setting can better capture its underlying representation.
  arXiv  Detail & Related papers  (2021-10-08T20:11:09Z)
• TABBIE: Pretrained Representations of Tabular Data [22.444607481407633]
  We devise a simple pretraining objective that learns exclusively from tabular data. Unlike competing approaches, our model (TABBIE) provides embeddings of all table substructures. A qualitative analysis of our model's learned cell, column, and row representations shows that it understands complex table semantics and numerical trends.
  arXiv  Detail & Related papers  (2021-05-06T11:15:16Z)
• TCN: Table Convolutional Network for Web Table Interpretation [52.32515851633981]
  We propose a novel table representation learning approach considering both the intra- and inter-table contextual information. Our method can outperform competitive baselines by +4.8% of F1 for column type prediction and by +4.1% of F1 for column pairwise relation prediction.
  arXiv  Detail & Related papers  (2021-02-17T02:18:10Z)
• GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing [117.98107557103877]
  We present GraPPa, an effective pre-training approach for table semantic parsing. We construct synthetic question-pairs over high-free tables via a synchronous context-free grammar. To maintain the model's ability to represent real-world data, we also include masked language modeling.
  arXiv  Detail & Related papers  (2020-09-29T08:17:58Z)
• TURL: Table Understanding through Representation Learning [29.6016859927782]
  TURL is a novel framework that introduces the pre-training/finetuning paradigm to relational Web tables. During pre-training, our framework learns deep contextualized representations on relational tables in an unsupervised manner. We show that TURL generalizes well to all tasks and substantially outperforms existing methods in almost all instances.
  arXiv  Detail & Related papers  (2020-06-26T05:44:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.