Doc-GCN: Heterogeneous Graph Convolutional Networks for Document Layout Analysis

• URL: http://arxiv.org/abs/2208.10970v1
• Date: Mon, 22 Aug 2022 07:22:05 GMT
• Title: Doc-GCN: Heterogeneous Graph Convolutional Networks for Document Layout Analysis
• Authors: Siwen Luo, Yihao Ding, Siqu Long, Soyeon Caren Han, Josiah Poon
• Abstract summary: Our Doc-GCN presents an effective way to harmonize and integrate heterogeneous aspects for Document Layout Analysis. We first construct graphs to explicitly describe four main aspects, including syntactic, semantic, density, and appearance/visual information. We apply graph convolutional networks for representing each aspect of information and use pooling to integrate them.
• Score: 4.920817773181236
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recognizing the layout of unstructured digital documents is crucial when parsing the documents into the structured, machine-readable format for downstream applications. Recent studies in Document Layout Analysis usually rely on computer vision models to understand documents while ignoring other information, such as context information or relation of document components, which are vital to capture. Our Doc-GCN presents an effective way to harmonize and integrate heterogeneous aspects for Document Layout Analysis. We first construct graphs to explicitly describe four main aspects, including syntactic, semantic, density, and appearance/visual information. Then, we apply graph convolutional networks for representing each aspect of information and use pooling to integrate them. Finally, we aggregate each aspect and feed them into 2-layer MLPs for document layout component classification. Our Doc-GCN achieves new state-of-the-art results in three widely used DLA datasets.

Related papers

• Unified Multi-Modal Interleaved Document Representation for Information Retrieval [57.65409208879344]
  We produce more comprehensive and nuanced document representations by holistically embedding documents interleaved with different modalities. Specifically, we achieve this by leveraging the capability of recent vision-language models that enable the processing and integration of text, images, and tables into a unified format and representation.
  arXiv  Detail & Related papers  (2024-10-03T17:49:09Z)
• DocGenome: An Open Large-scale Scientific Document Benchmark for Training and Testing Multi-modal Large Language Models [63.466265039007816]
  We present DocGenome, a structured document benchmark constructed by annotating 500K scientific documents from 153 disciplines in the arXiv open-access community. We conduct extensive experiments to demonstrate the advantages of DocGenome and objectively evaluate the performance of large models on our benchmark.
  arXiv  Detail & Related papers  (2024-06-17T15:13:52Z)
• Enhancing Visually-Rich Document Understanding via Layout Structure Modeling [91.07963806829237]
  We propose GraphLM, a novel document understanding model that injects layout knowledge into the model. We evaluate our model on various benchmarks, including FUNSD, XFUND and CORD, and achieve state-of-the-art results.
  arXiv  Detail & Related papers  (2023-08-15T13:53:52Z)
• Unifying Vision, Text, and Layout for Universal Document Processing [105.36490575974028]
  We propose a Document AI model which unifies text, image, and layout modalities together with varied task formats, including document understanding and generation. Our method sets the state-of-the-art on 9 Document AI tasks, e.g., document understanding and QA, across diverse data domains like finance reports, academic papers, and websites.
  arXiv  Detail & Related papers  (2022-12-05T22:14:49Z)
• Unified Pretraining Framework for Document Understanding [52.224359498792836]
  We present UDoc, a new unified pretraining framework for document understanding. UDoc is designed to support most document understanding tasks, extending the Transformer to take multimodal embeddings as input. An important feature of UDoc is that it learns a generic representation by making use of three self-supervised losses.
  arXiv  Detail & Related papers  (2022-04-22T21:47:04Z)
• Multimodal Pre-training Based on Graph Attention Network for Document Understanding [32.55734039518983]
  GraphDoc is a graph-based model for various document understanding tasks. It is pre-trained in a multimodal framework by utilizing text, layout, and image information simultaneously. It learns a generic representation from only 320k unlabeled documents.
  arXiv  Detail & Related papers  (2022-03-25T09:27:50Z)
• Cross-Domain Document Layout Analysis Using Document Style Guide [15.799572801059716]
  Document layout analysis (DLA) aims to decompose document images into high-level semantic areas. Many researchers devoted this challenge by synthesizing data to build large training sets. In this paper, we propose an unsupervised cross-domain DLA framework based on document style guidance.
  arXiv  Detail & Related papers  (2022-01-24T00:49:19Z)
• Synthetic Document Generator for Annotation-free Layout Recognition [15.657295650492948]
  We describe a synthetic document generator that automatically produces realistic documents with labels for spatial positions, extents and categories of layout elements. We empirically illustrate that a deep layout detection model trained purely on the synthetic documents can match the performance of a model that uses real documents.
  arXiv  Detail & Related papers  (2021-11-11T01:58:44Z)
• Evaluation of a Region Proposal Architecture for Multi-task Document Layout Analysis [0.685316573653194]
  Mask-RCNN architecture is designed to address the problem of baseline detection and region segmentation. We present experimental results on two handwritten text datasets and one handwritten music dataset. The analyzed architecture yields promising results, outperforming state-of-the-art techniques in all three datasets.
  arXiv  Detail & Related papers  (2021-06-22T14:07:27Z)
• VSR: A Unified Framework for Document Layout Analysis combining Vision, Semantics and Relations [40.721146438291335]
  We propose a unified framework VSR for document layout analysis, combining vision, semantics and relations. On three popular benchmarks, VSR outperforms previous models by large margins.
  arXiv  Detail & Related papers  (2021-05-13T12:20:30Z)
• DocBank: A Benchmark Dataset for Document Layout Analysis [114.81155155508083]
  We present textbfDocBank, a benchmark dataset that contains 500K document pages with fine-grained token-level annotations for document layout analysis. Experiment results show that models trained on DocBank accurately recognize the layout information for a variety of documents.
  arXiv  Detail & Related papers  (2020-06-01T16:04:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.