SRFUND: A Multi-Granularity Hierarchical Structure Reconstruction Benchmark in Form Understanding

• URL: http://arxiv.org/abs/2406.08757v1
• Date: Thu, 13 Jun 2024 02:35:55 GMT
• Title: SRFUND: A Multi-Granularity Hierarchical Structure Reconstruction Benchmark in Form Understanding
• Authors: Jiefeng Ma, Yan Wang, Chenyu Liu, Jun Du, Yu Hu, Zhenrong Zhang, Pengfei Hu, Qing Wang, Jianshu Zhang,
• Abstract summary: We present the SRFUND, a hierarchically structured multi-task form understanding benchmark. SRFUND provides refined annotations on top of the original FUNSD and XFUND datasets. The dataset includes eight languages including English, Chinese, Japanese, German, French, Spanish, Italian, and Portuguese.
• Score: 55.48936731641802
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Accurately identifying and organizing textual content is crucial for the automation of document processing in the field of form understanding. Existing datasets, such as FUNSD and XFUND, support entity classification and relationship prediction tasks but are typically limited to local and entity-level annotations. This limitation overlooks the hierarchically structured representation of documents, constraining comprehensive understanding of complex forms. To address this issue, we present the SRFUND, a hierarchically structured multi-task form understanding benchmark. SRFUND provides refined annotations on top of the original FUNSD and XFUND datasets, encompassing five tasks: (1) word to text-line merging, (2) text-line to entity merging, (3) entity category classification, (4) item table localization, and (5) entity-based full-document hierarchical structure recovery. We meticulously supplemented the original dataset with missing annotations at various levels of granularity and added detailed annotations for multi-item table regions within the forms. Additionally, we introduce global hierarchical structure dependencies for entity relation prediction tasks, surpassing traditional local key-value associations. The SRFUND dataset includes eight languages including English, Chinese, Japanese, German, French, Spanish, Italian, and Portuguese, making it a powerful tool for cross-lingual form understanding. Extensive experimental results demonstrate that the SRFUND dataset presents new challenges and significant opportunities in handling diverse layouts and global hierarchical structures of forms, thus providing deep insights into the field of form understanding. The original dataset and implementations of baseline methods are available at https://sprateam-ustc.github.io/SRFUND

Related papers

• Multi-Field Adaptive Retrieval [39.38972160512916]
  We introduce Multi-Field Adaptive Retrieval (MFAR), a flexible framework that accommodates any number of document indices on structured data. Our framework consists of two main steps: (1) the decomposition of an existing document into fields, each indexed independently through dense and lexical methods, and (2) learning a model which adaptively predicts the importance of a field by conditioning on the document query. We find that our approach allows for the optimized use of dense versus lexical representations across field types, significantly improves in document ranking over a number of existing retrievers, and achieves state-of-the-art performance for multi-field structured
  arXiv  Detail & Related papers  (2024-10-26T03:07:22Z)
• MGSA: Multi-Granularity Graph Structure Attention for Knowledge Graph-to-Text Generation [10.607080796475815]
  This paper introduces the Multi-granularity Graph Structure Attention (MGSA), which is based on pre-trained language models (PLMs) The encoder of the model architecture features an entity-level structure encoding module, a word-level structure encoding module, and an aggregation module that synthesizes information from both structure. We conducted extensive evaluations of the MGSA model using two widely recognized KG-to-Text Generation benchmark datasets, WebNLG and EventNarrative.
  arXiv  Detail & Related papers  (2024-09-16T14:01:03Z)
• XFormParser: A Simple and Effective Multimodal Multilingual Semi-structured Form Parser [35.69888780388425]
  In this work, we introduce a simple but effective textbfMultimodal and textbfMultilingual semi-structured textbfFORM textbfXForm framework. textbfXForm is anchored on a comprehensive pre-trained language model and innovatively amalgamates entity recognition and relationRE. Our framework exhibits exceptionally improved performance across tasks in both multi-language and zero-shot contexts.
  arXiv  Detail & Related papers  (2024-05-27T16:37:17Z)
• mPLUG-DocOwl 1.5: Unified Structure Learning for OCR-free Document Understanding [100.17063271791528]
  We propose the Unified Structure Learning to boost the performance of MLLMs. Our model DocOwl 1.5 achieves state-of-the-art performance on 10 visual document understanding benchmarks.
  arXiv  Detail & Related papers  (2024-03-19T16:48:40Z)
• DIVKNOWQA: Assessing the Reasoning Ability of LLMs via Open-Domain Question Answering over Knowledge Base and Text [73.68051228972024]
  Large Language Models (LLMs) have exhibited impressive generation capabilities, but they suffer from hallucinations when relying on their internal knowledge. Retrieval-augmented LLMs have emerged as a potential solution to ground LLMs in external knowledge.
  arXiv  Detail & Related papers  (2023-10-31T04:37:57Z)
• DocTr: Document Transformer for Structured Information Extraction in Documents [36.1145541816468]
  We present a new formulation for structured information extraction from visually rich documents. It aims to address the limitations of existing IOB tagging or graph-based formulations. We represent an entity as an anchor word and a bounding box, and represent entity linking as the association between anchor words.
  arXiv  Detail & Related papers  (2023-07-16T02:59:30Z)
• Pre-training Language Model Incorporating Domain-specific Heterogeneous Knowledge into A Unified Representation [49.89831914386982]
  We propose a unified pre-trained language model (PLM) for all forms of text, including unstructured text, semi-structured text, and well-structured text. Our approach outperforms the pre-training of plain text using only 1/4 of the data.
  arXiv  Detail & Related papers  (2021-09-02T16:05:24Z)
• StrucTexT: Structured Text Understanding with Multi-Modal Transformers [29.540122964399046]
  Structured text understanding on Visually Rich Documents (VRDs) is a crucial part of Document Intelligence. This paper proposes a unified framework named StrucTexT, which is flexible and effective for handling both sub-tasks. We evaluate our method for structured text understanding at segment-level and token-level and show it outperforms the state-of-the-art counterparts.
  arXiv  Detail & Related papers  (2021-08-06T02:57:07Z)
• Minimally-Supervised Structure-Rich Text Categorization via Learning on Text-Rich Networks [61.23408995934415]
  We propose a novel framework for minimally supervised categorization by learning from the text-rich network. Specifically, we jointly train two modules with different inductive biases -- a text analysis module for text understanding and a network learning module for class-discriminative, scalable network learning. Our experiments show that given only three seed documents per category, our framework can achieve an accuracy of about 92%.
  arXiv  Detail & Related papers  (2021-02-23T04:14:34Z)
• DART: Open-Domain Structured Data Record to Text Generation [91.23798751437835]
  We present DART, an open domain structured DAta Record to Text generation dataset with over 82k instances (DARTs) We propose a procedure of extracting semantic triples from tables that encode their structures by exploiting the semantic dependencies among table headers and the table title. Our dataset construction framework effectively merged heterogeneous sources from open domain semantic parsing and dialogue-act-based meaning representation tasks.
  arXiv  Detail & Related papers  (2020-07-06T16:35:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.