Layout-Aware Information Extraction for Document-Grounded Dialogue: Dataset, Method and Demonstration

• URL: http://arxiv.org/abs/2207.06717v1
• Date: Thu, 14 Jul 2022 07:59:45 GMT
• Title: Layout-Aware Information Extraction for Document-Grounded Dialogue: Dataset, Method and Demonstration
• Authors: Zhenyu Zhang, Bowen Yu, Haiyang Yu, Tingwen Liu, Cheng Fu, Jingyang Li, Chengguang Tang, Jian Sun, Yongbin Li
• Abstract summary: We propose a layout-aware document-level Information Extraction dataset, LIE, to facilitate the study of extracting both structural and semantic knowledge from visually rich documents. LIE contains 62k annotations of three extraction tasks from 4,061 pages in product and official documents. Empirical results show that layout is critical for VRD-based extraction, and system demonstration also verifies that the extracted knowledge can help locate the answers that users care about.
• Score: 75.47708732473586
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Building document-grounded dialogue systems have received growing interest as documents convey a wealth of human knowledge and commonly exist in enterprises. Wherein, how to comprehend and retrieve information from documents is a challenging research problem. Previous work ignores the visual property of documents and treats them as plain text, resulting in incomplete modality. In this paper, we propose a Layout-aware document-level Information Extraction dataset, LIE, to facilitate the study of extracting both structural and semantic knowledge from visually rich documents (VRDs), so as to generate accurate responses in dialogue systems. LIE contains 62k annotations of three extraction tasks from 4,061 pages in product and official documents, becoming the largest VRD-based information extraction dataset to the best of our knowledge. We also develop benchmark methods that extend the token-based language model to consider layout features like humans. Empirical results show that layout is critical for VRD-based extraction, and system demonstration also verifies that the extracted knowledge can help locate the answers that users care about.

Related papers

• BoundingDocs: a Unified Dataset for Document Question Answering with Spatial Annotations [2.9798896492745537]
  We present a unified dataset for document Question-Answering (QA) We reformulate existing Document AI tasks, such as Information Extraction (IE), into a Question-Answering task. On the other hand, we release the OCR of all the documents and include the exact position of the answer to be found in the document image as a bounding box.
  arXiv  Detail & Related papers  (2025-01-06T21:46:22Z)
• DocKD: Knowledge Distillation from LLMs for Open-World Document Understanding Models [66.91204604417912]
  This study aims to enhance generalizability of small VDU models by distilling knowledge from LLMs. We present a new framework (called DocKD) that enriches the data generation process by integrating external document knowledge. Experiments show that DocKD produces high-quality document annotations and surpasses the direct knowledge distillation approach.
  arXiv  Detail & Related papers  (2024-10-04T00:53:32Z)
• Unified Multimodal Interleaved Document Representation for Retrieval [57.65409208879344]
  We propose a method that holistically embeds documents interleaved with multiple modalities. We merge the representations of segmented passages into one single document representation. We show that our approach substantially outperforms relevant baselines.
  arXiv  Detail & Related papers  (2024-10-03T17:49:09Z)
• Coarse-to-Fine Knowledge Selection for Document Grounded Dialogs [11.63334863772068]
  Multi-document grounded dialogue systems (DGDS) answer users' requests by finding supporting knowledge from a collection of documents. This paper proposes Re3G, which aims to optimize both coarse-grained knowledge retrieval and fine-grained knowledge extraction in a unified framework.
  arXiv  Detail & Related papers  (2023-02-23T08:28:29Z)
• TRIE++: Towards End-to-End Information Extraction from Visually Rich Documents [51.744527199305445]
  This paper proposes a unified end-to-end information extraction framework from visually rich documents. Text reading and information extraction can reinforce each other via a well-designed multi-modal context block. The framework can be trained in an end-to-end trainable manner, achieving global optimization.
  arXiv  Detail & Related papers  (2022-07-14T08:52:07Z)
• Unified Pretraining Framework for Document Understanding [52.224359498792836]
  We present UDoc, a new unified pretraining framework for document understanding. UDoc is designed to support most document understanding tasks, extending the Transformer to take multimodal embeddings as input. An important feature of UDoc is that it learns a generic representation by making use of three self-supervised losses.
  arXiv  Detail & Related papers  (2022-04-22T21:47:04Z)
• Combining Deep Learning and Reasoning for Address Detection in Unstructured Text Documents [0.0]
  We propose a hybrid approach that combines deep learning with reasoning for finding and extracting addresses from unstructured text documents. We use a visual deep learning model to detect the boundaries of possible address regions on the scanned document images.
  arXiv  Detail & Related papers  (2022-02-07T12:32:00Z)
• Evaluation of a Region Proposal Architecture for Multi-task Document Layout Analysis [0.685316573653194]
  Mask-RCNN architecture is designed to address the problem of baseline detection and region segmentation. We present experimental results on two handwritten text datasets and one handwritten music dataset. The analyzed architecture yields promising results, outperforming state-of-the-art techniques in all three datasets.
  arXiv  Detail & Related papers  (2021-06-22T14:07:27Z)
• TRIE: End-to-End Text Reading and Information Extraction for Document Understanding [56.1416883796342]
  We propose a unified end-to-end text reading and information extraction network. multimodal visual and textual features of text reading are fused for information extraction. Our proposed method significantly outperforms the state-of-the-art methods in both efficiency and accuracy.
  arXiv  Detail & Related papers  (2020-05-27T01:47:26Z)
• Natural language processing for word sense disambiguation and information extraction [0.0]
  The thesis presents a new approach for Word Sense Disambiguation using thesaurus. A Document Retrieval method, based on Fuzzy Logic has been described and its application is illustrated. The strategy concludes with the presentation of a novel strategy based on Dempster-Shafer theory of evidential reasoning.
  arXiv  Detail & Related papers  (2020-04-05T17:13:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.