Multi-Cell Decoder and Mutual Learning for Table Structure and Character Recognition

• URL: http://arxiv.org/abs/2404.13268v2
• Date: Sun, 12 May 2024 15:11:25 GMT
• Title: Multi-Cell Decoder and Mutual Learning for Table Structure and Character Recognition
• Authors: Takaya Kawakatsu,
• Abstract summary: We propose a multi-cell content decoder and bidirectional mutual learning mechanism to improve the end-to-end approach. The effectiveness is demonstrated on two large datasets, and the experimental results show comparable performance to state-of-the-art models.
• Score: 1.2328446298523066
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Extracting table contents from documents such as scientific papers and financial reports and converting them into a format that can be processed by large language models is an important task in knowledge information processing. End-to-end approaches, which recognize not only table structure but also cell contents, achieved performance comparable to state-of-the-art models using external character recognition systems, and have potential for further improvements. In addition, these models can now recognize long tables with hundreds of cells by introducing local attention. However, the models recognize table structure in one direction from the header to the footer, and cell content recognition is performed independently for each cell, so there is no opportunity to retrieve useful information from the neighbor cells. In this paper, we propose a multi-cell content decoder and bidirectional mutual learning mechanism to improve the end-to-end approach. The effectiveness is demonstrated on two large datasets, and the experimental results show comparable performance to state-of-the-art models, even for long tables with large numbers of cells.

Related papers

• UniTabNet: Bridging Vision and Language Models for Enhanced Table Structure Recognition [55.153629718464565]
  We introduce UniTabNet, a novel framework for table structure parsing based on the image-to-text model. UniTabNet employs a divide-and-conquer'' strategy, utilizing an image-to-text model to decouple table cells and integrating both physical and logical decoders to reconstruct the complete table structure.
  arXiv  Detail & Related papers  (2024-09-20T01:26:32Z)
• Knowledge-Aware Reasoning over Multimodal Semi-structured Tables [85.24395216111462]
  This study investigates whether current AI models can perform knowledge-aware reasoning on multimodal structured data. We introduce MMTabQA, a new dataset designed for this purpose. Our experiments highlight substantial challenges for current AI models in effectively integrating and interpreting multiple text and image inputs.
  arXiv  Detail & Related papers  (2024-08-25T15:17:43Z)
• UniCell: Universal Cell Nucleus Classification via Prompt Learning [76.11864242047074]
  We propose a universal cell nucleus classification framework (UniCell) It employs a novel prompt learning mechanism to uniformly predict the corresponding categories of pathological images from different dataset domains. In particular, our framework adopts an end-to-end architecture for nuclei detection and classification, and utilizes flexible prediction heads for adapting various datasets.
  arXiv  Detail & Related papers  (2024-02-20T11:50:27Z)
• An End-to-End Multi-Task Learning Model for Image-based Table Recognition [4.530704014707227]
  We propose an end-to-end multi-task learning model for image-based table recognition. The proposed model consists of one shared encoder, one shared decoder, and three separate decoders. The whole system can be easily trained and inferred in an end-to-end approach.
  arXiv  Detail & Related papers  (2023-03-15T14:24:01Z)
• TRUST: An Accurate and End-to-End Table structure Recognizer Using Splitting-based Transformers [56.56591337457137]
  We propose an accurate and end-to-end transformer-based table structure recognition method, referred to as TRUST. Transformers are suitable for table structure recognition because of their global computations, perfect memory, and parallel computation. We conduct experiments on several popular benchmarks including PubTabNet and SynthTable, our method achieves new state-of-the-art results.
  arXiv  Detail & Related papers  (2022-08-31T08:33:36Z)
• Table Structure Recognition with Conditional Attention [13.976736586808308]
  Table Structure Recognition (TSR) problem aims to recognize the structure of a table and transform the unstructured tables into a structured and machine-readable format. In this study, we hypothesize that a complicated table structure can be represented by a graph whose vertices and edges represent the cells and association between cells, respectively. Experimental results show that the alignment of a cell bounding box can help improve the Micro-averaged F1 score from 0.915 to 0.963, and the Macro-average F1 score from 0.787 to 0.923.
  arXiv  Detail & Related papers  (2022-03-08T02:44:58Z)
• Visual Understanding of Complex Table Structures from Document Images [32.95187519339354]
  We propose a novel object-detection-based deep model that captures the inherent alignments of cells within tables. We also aim to improve structure recognition by deducing a novel rectilinear graph-based formulation. Our framework improves the previous state-of-the-art performance by a 2.7% average F1-score on benchmark datasets.
  arXiv  Detail & Related papers  (2021-11-13T14:54:33Z)
• Split, embed and merge: An accurate table structure recognizer [42.579215135672094]
  We introduce Split, Embed and Merge (SEM) as an accurate table structure recognizer. SEM can achieve an average F-Measure of $96.9%$ on the SciTSR dataset.
  arXiv  Detail & Related papers  (2021-07-12T06:26:19Z)
• TGRNet: A Table Graph Reconstruction Network for Table Structure Recognition [76.06530816349763]
  We propose an end-to-end trainable table graph reconstruction network (TGRNet) for table structure recognition. Specifically, the proposed method has two main branches, a cell detection branch and a cell logical location branch, to jointly predict the spatial location and the logical location of different cells.
  arXiv  Detail & Related papers  (2021-06-20T01:57:05Z)
• TCN: Table Convolutional Network for Web Table Interpretation [52.32515851633981]
  We propose a novel table representation learning approach considering both the intra- and inter-table contextual information. Our method can outperform competitive baselines by +4.8% of F1 for column type prediction and by +4.1% of F1 for column pairwise relation prediction.
  arXiv  Detail & Related papers  (2021-02-17T02:18:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.