Relational Representation Learning in Visually-Rich Documents

• URL: http://arxiv.org/abs/2205.02411v1
• Date: Thu, 5 May 2022 02:54:31 GMT
• Title: Relational Representation Learning in Visually-Rich Documents
• Abstract summary: We propose DocReL, a contextual Document Representation framework. The major challenge of DocReL roots in the variety of relations. We propose a novel contrastive task named Consistency Modeling (RCM), which harnesses the fact that existing relations should be consistent in augmented positive views. RCM provides relational representations which are more compatible to the urgent need of downstream tasks, even without any knowledge about the exact definition of relation.
• Score: 14.942600317107082
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Relational understanding is critical for a number of visually-rich documents (VRDs) understanding tasks. Through multi-modal pre-training, recent studies provide comprehensive contextual representations and exploit them as prior knowledge for downstream tasks. In spite of their impressive results, we observe that the widespread relational hints (e.g., relation of key/value fields on receipts) built upon contextual knowledge are not excavated yet. To mitigate this gap, we propose DocReL, a Document Relational Representation Learning framework. The major challenge of DocReL roots in the variety of relations. From the simplest pairwise relation to the complex global structure, it is infeasible to conduct supervised training due to the definition of relation varies and even conflicts in different tasks. To deal with the unpredictable definition of relations, we propose a novel contrastive learning task named Relational Consistency Modeling (RCM), which harnesses the fact that existing relations should be consistent in differently augmented positive views. RCM provides relational representations which are more compatible to the urgent need of downstream tasks, even without any knowledge about the exact definition of relation. DocReL achieves better performance on a wide variety of VRD relational understanding tasks, including table structure recognition, key information extraction and reading order detection.

Related papers

• Reasoning-enhanced Query Understanding through Decomposition and Interpretation [87.56450566014625]
  ReDI is a Reasoning-enhanced approach for query understanding through Decomposition and Interpretation.<n>We compiled a large-scale dataset of real-world complex queries from a major search engine.<n> Experiments on BRIGHT and BEIR demonstrate that ReDI consistently surpasses strong baselines in both sparse and dense retrieval paradigms.
  arXiv  Detail & Related papers  (2025-09-08T10:58:42Z)
• ART: Adaptive Relation Tuning for Generalized Relation Prediction [33.15138052099355]
  Visual relation detection (VRD) is the task of identifying the relationships between objects in a scene.<n>While prompt tuning has been used to adapt vision-language models (VLMs) for VRD, it uses handcrafted prompts and struggles with novel or complex relations.<n>We argue that instruction tuning offers a more effective solution by fine-tuning VLMs on diverse instructional data.
  arXiv  Detail & Related papers  (2025-07-31T13:34:06Z)
• Navigating Semantic Relations: Challenges for Language Models in Abstract Common-Sense Reasoning [5.4141465747474475]
  Large language models (LLMs) have achieved remarkable performance in generating human-like text and solving problems of moderate complexity. We systematically evaluate abstract common-sense reasoning in LLMs using the ConceptNet knowledge graph.
  arXiv  Detail & Related papers  (2025-02-19T20:20:24Z)
• Zero-Shot Relational Learning for Multimodal Knowledge Graphs [31.215889061734295]
  One of the major challenges is inference on newly discovered relations without associated training data. Existing works fail to support the leverage of multimodal information and leave the problem unexplored. We propose a novel end-to-end framework, consisting of three components, i.e., multimodal learner, structure consolidator embedding generator, to integrate diverse multimodal information and knowledge graph structures.
  arXiv  Detail & Related papers  (2024-04-09T11:14:45Z)
• Multiple Relations Classification using Imbalanced Predictions Adaptation [0.0]
  The relation classification task assigns the proper semantic relation to a pair of subject and object entities. Current relation classification models employ additional procedures to identify multiple relations in a single sentence. We propose a multiple relations classification model that tackles these issues through a customized output architecture and by exploiting additional input features.
  arXiv  Detail & Related papers  (2023-09-24T18:36:22Z)
• Document-level Relation Extraction with Relation Correlations [15.997345900917058]
  Document-level relation extraction faces two overlooked challenges: long-tail problem and multi-label problem. We analyze the co-occurrence correlation of relations, and introduce it into DocRE task for the first time.
  arXiv  Detail & Related papers  (2022-12-20T11:17:52Z)
• ViRel: Unsupervised Visual Relations Discovery with Graph-level Analogy [65.5580334698777]
  ViRel is a method for unsupervised discovery and learning of Visual Relations with graph-level analogy. We show that our method achieves above 95% accuracy in relation classification. We further generalizes to unseen tasks with more complicated relational structures.
  arXiv  Detail & Related papers  (2022-07-04T16:56:45Z)
• Saliency-Regularized Deep Multi-Task Learning [7.3810864598379755]
  Multitask learning enforces multiple learning tasks to share knowledge to improve their generalization abilities. Modern deep multitask learning can jointly learn latent features and task sharing, but they are obscure in task relation. This paper proposes a new multitask learning framework that jointly learns latent features and explicit task relations.
  arXiv  Detail & Related papers  (2022-07-03T20:26:44Z)
• Improving Long Tailed Document-Level Relation Extraction via Easy Relation Augmentation and Contrastive Learning [66.83982926437547]
  We argue that mitigating the long-tailed distribution problem is crucial for DocRE in the real-world scenario. Motivated by the long-tailed distribution problem, we propose an Easy Relation Augmentation(ERA) method for improving DocRE.
  arXiv  Detail & Related papers  (2022-05-21T06:15:11Z)
• ZS-BERT: Towards Zero-Shot Relation Extraction with Attribute Representation Learning [10.609715843964263]
  We formulate the zero-shot relation extraction problem by incorporating the text description of seen and unseen relations. We propose a novel multi-task learning model, zero-shot BERT, to directly predict unseen relations without hand-crafted labeling and multiple pairwise attribute classifications. Experiments conducted on two well-known datasets exhibit that ZS-BERT can outperform existing methods by at least 13.54% improvement on F1 score.
  arXiv  Detail & Related papers  (2021-04-10T06:53:41Z)
• ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learning [97.10875695679499]
  We propose a novel contrastive learning framework named ERICA in pre-training phase to obtain a deeper understanding of the entities and their relations in text. Experimental results demonstrate that our proposed ERICA framework achieves consistent improvements on several document-level language understanding tasks.
  arXiv  Detail & Related papers  (2020-12-30T03:35:22Z)
• Learning Relation Prototype from Unlabeled Texts for Long-tail Relation Extraction [84.64435075778988]
  We propose a general approach to learn relation prototypes from unlabeled texts. We learn relation prototypes as an implicit factor between entities. We conduct experiments on two publicly available datasets: New York Times and Google Distant Supervision.
  arXiv  Detail & Related papers  (2020-11-27T06:21:12Z)
• Learning to Decouple Relations: Few-Shot Relation Classification with Entity-Guided Attention and Confusion-Aware Training [49.9995628166064]
  We propose CTEG, a model equipped with two mechanisms to learn to decouple easily-confused relations. On the one hand, an EGA mechanism is introduced to guide the attention to filter out information causing confusion. On the other hand, a Confusion-Aware Training (CAT) method is proposed to explicitly learn to distinguish relations.
  arXiv  Detail & Related papers  (2020-10-21T11:07:53Z)
• Improving Long-Tail Relation Extraction with Collaborating Relation-Augmented Attention [63.26288066935098]
  We propose a novel neural network, Collaborating Relation-augmented Attention (CoRA), to handle both the wrong labeling and long-tail relations. In the experiments on the popular benchmark dataset NYT, the proposed CoRA improves the prior state-of-the-art performance by a large margin.
  arXiv  Detail & Related papers  (2020-10-08T05:34:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.