TabXEval: Why this is a Bad Table? An eXhaustive Rubric for Table Evaluation

• URL: http://arxiv.org/abs/2505.22176v2
• Date: Sun, 01 Jun 2025 15:12:03 GMT
• Title: TabXEval: Why this is a Bad Table? An eXhaustive Rubric for Table Evaluation
• Authors: Vihang Pancholi, Jainit Bafna, Tejas Anvekar, Manish Shrivastava, Vivek Gupta,
• Abstract summary: We propose a rubric-based evaluation framework that integrates multi-level structural descriptors with fine-grained contextual signals.<n>We introduce TabXEval, an eXhaustive and eXplainable two-phase evaluation framework.<n>We evaluate TabXEval on TabXBench, a diverse, multi-domain benchmark featuring realistic table perturbations and human annotations.
• Score: 10.212570261759204
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Evaluating tables qualitatively and quantitatively poses a significant challenge, as standard metrics often overlook subtle structural and content-level discrepancies. To address this, we propose a rubric-based evaluation framework that integrates multi-level structural descriptors with fine-grained contextual signals, enabling more precise and consistent table comparison. Building on this, we introduce TabXEval, an eXhaustive and eXplainable two-phase evaluation framework. TabXEval first aligns reference and predicted tables structurally via TabAlign, then performs semantic and syntactic comparison using TabCompare, offering interpretable and granular feedback. We evaluate TabXEval on TabXBench, a diverse, multi-domain benchmark featuring realistic table perturbations and human annotations. A sensitivity-specificity analysis further demonstrates the robustness and explainability of TabXEval across varied table tasks. Code and data are available at https://coral-lab-asu.github.io/tabxeval/

Related papers

• TableEval: A Real-World Benchmark for Complex, Multilingual, and Multi-Structured Table Question Answering [18.173773939709733]
  Existing TableQA benchmarks are often limited by their focus on simple flat tables and suffer from data leakage.<n>We introduce TableEval, a new benchmark designed to evaluate LLMs on realistic TableQA tasks.<n>To minimize the risk of data leakage, we collect all data from recent real-world documents.
  arXiv  Detail & Related papers  (2025-06-04T13:39:01Z)
• ArxivDIGESTables: Synthesizing Scientific Literature into Tables using Language Models [58.34560740973768]
  We introduce a framework that leverages language models (LMs) to generate literature review tables. A new dataset of 2,228 literature review tables extracted from ArXiv papers synthesize a total of 7,542 research papers. We evaluate LMs' abilities to reconstruct reference tables, finding this task benefits from additional context.
  arXiv  Detail & Related papers  (2024-10-25T18:31:50Z)
• UniTabNet: Bridging Vision and Language Models for Enhanced Table Structure Recognition [55.153629718464565]
  We introduce UniTabNet, a novel framework for table structure parsing based on the image-to-text model. UniTabNet employs a divide-and-conquer'' strategy, utilizing an image-to-text model to decouple table cells and integrating both physical and logical decoders to reconstruct the complete table structure.
  arXiv  Detail & Related papers  (2024-09-20T01:26:32Z)
• Top-K Pairwise Ranking: Bridging the Gap Among Ranking-Based Measures for Multi-Label Classification [120.37051160567277]
  This paper proposes a novel measure named Top-K Pairwise Ranking (TKPR) A series of analyses show that TKPR is compatible with existing ranking-based measures. On the other hand, we establish a sharp generalization bound for the proposed framework based on a novel technique named data-dependent contraction.
  arXiv  Detail & Related papers  (2024-07-09T09:36:37Z)
• LaTable: Towards Large Tabular Models [63.995130144110156]
  Tabular generative foundation models are hard to build due to the heterogeneous feature spaces of different datasets. LaTable is a novel diffusion model that addresses these challenges and can be trained across different datasets. We find that LaTable outperforms baselines on in-distribution generation, and that finetuning LaTable can generate out-of-distribution datasets better with fewer samples.
  arXiv  Detail & Related papers  (2024-06-25T16:03:50Z)
• Is This a Bad Table? A Closer Look at the Evaluation of Table Generation from Text [21.699434525769586]
  Existing measures for table quality evaluation fail to capture the overall semantics of the tables. We propose TabEval, a novel table evaluation strategy that captures table semantics. To validate our approach, we curate a dataset comprising of text descriptions for 1,250 diverse Wikipedia tables.
  arXiv  Detail & Related papers  (2024-06-21T02:18:03Z)
• UniTabE: A Universal Pretraining Protocol for Tabular Foundation Model in Data Science [16.384705926693073]
  This study seeks to extend the power of pretraining methodologies to facilitate the prediction over tables in data science. We introduce UniTabE, a method designed to process tables in a uniform manner, devoid of constraints imposed by specific table structures. In order to implement the pretraining phase, we curated an expansive dataset comprising approximately 13B samples, meticulously gathered from the Kaggle platform.
  arXiv  Detail & Related papers  (2023-07-18T13:28:31Z)
• SEMv2: Table Separation Line Detection Based on Instance Segmentation [96.36188168694781]
  We propose an accurate table structure recognizer, termed SEMv2 (SEM: Split, Embed and Merge) We address the table separation line instance-level discrimination problem and introduce a table separation line detection strategy based on conditional convolution. To comprehensively evaluate the SEMv2, we also present a more challenging dataset for table structure recognition, dubbed iFLYTAB.
  arXiv  Detail & Related papers  (2023-03-08T05:15:01Z)
• TRUST: An Accurate and End-to-End Table structure Recognizer Using Splitting-based Transformers [56.56591337457137]
  We propose an accurate and end-to-end transformer-based table structure recognition method, referred to as TRUST. Transformers are suitable for table structure recognition because of their global computations, perfect memory, and parallel computation. We conduct experiments on several popular benchmarks including PubTabNet and SynthTable, our method achieves new state-of-the-art results.
  arXiv  Detail & Related papers  (2022-08-31T08:33:36Z)
• TableFormer: Robust Transformer Modeling for Table-Text Encoding [18.00127368618485]
  Existing models for table understanding require linearization of the table structure, where row or column order is encoded as an unwanted bias. In this work, we propose a robust and structurally aware table-text encoding architecture TableFormer.
  arXiv  Detail & Related papers  (2022-03-01T07:23:06Z)
• A Training-free and Reference-free Summarization Evaluation Metric via Centrality-weighted Relevance and Self-referenced Redundancy [60.419107377879925]
  We propose a training-free and reference-free summarization evaluation metric. Our metric consists of a centrality-weighted relevance score and a self-referenced redundancy score. Our methods can significantly outperform existing methods on both multi-document and single-document summarization evaluation.
  arXiv  Detail & Related papers  (2021-06-26T05:11:27Z)
• TGRNet: A Table Graph Reconstruction Network for Table Structure Recognition [76.06530816349763]
  We propose an end-to-end trainable table graph reconstruction network (TGRNet) for table structure recognition. Specifically, the proposed method has two main branches, a cell detection branch and a cell logical location branch, to jointly predict the spatial location and the logical location of different cells.
  arXiv  Detail & Related papers  (2021-06-20T01:57:05Z)
• Tab.IAIS: Flexible Table Recognition and Semantic Interpretation System [84.39812458417246]
  We develop two rule-based algorithms that perform the complete table recognition process and support the most frequent table formats. To incorporate the extraction of semantic information into the table recognition process, we develop a graph-based table interpretation method. Our table recognition approach achieves results competitive with state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-05-25T12:31:02Z)
• TabLeX: A Benchmark Dataset for Structure and Content Information Extraction from Scientific Tables [1.4115224153549193]
  This paper presents TabLeX, a large-scale benchmark dataset comprising table images generated from scientific articles. To facilitate the development of robust table IE tools, TabLeX contains images in different aspect ratios and in a variety of fonts. Our analysis sheds light on the shortcomings of current state-of-the-art table extraction models and shows that they fail on even simple table images.
  arXiv  Detail & Related papers  (2021-05-12T05:13:38Z)
• GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing [117.98107557103877]
  We present GraPPa, an effective pre-training approach for table semantic parsing. We construct synthetic question-pairs over high-free tables via a synchronous context-free grammar. To maintain the model's ability to represent real-world data, we also include masked language modeling.
  arXiv  Detail & Related papers  (2020-09-29T08:17:58Z)
• TabSim: A Siamese Neural Network for Accurate Estimation of Table Similarity [5.889134549635538]
  We present TabSim, a novel method to compute table similarity scores using deep neural networks. To train and evaluate our method, we created a gold standard corpus consisting of 1500 table pairs extracted from biomedical articles. Our evaluation shows that TabSim outperforms other table similarity measures on average by app. 7% pp F1-score in a binary similarity classification setting and by app. 1.5% pp in a ranking scenario.
  arXiv  Detail & Related papers  (2020-08-25T07:32:09Z)
• Towards Faithful Neural Table-to-Text Generation with Content-Matching Constraints [63.84063384518667]
  We propose a novel Transformer-based generation framework to achieve the goal. Core techniques in our method to enforce faithfulness include a new table-text optimal-transport matching loss. To evaluate faithfulness, we propose a new automatic metric specialized to the table-to-text generation problem.
  arXiv  Detail & Related papers  (2020-05-03T02:54:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.