ViRED: Prediction of Visual Relations in Engineering Drawings

• URL: http://arxiv.org/abs/2409.00909v1
• Date: Mon, 2 Sep 2024 02:42:34 GMT
• Title: ViRED: Prediction of Visual Relations in Engineering Drawings
• Authors: Chao Gu, Ke Lin, Yiyang Luo, Jiahui Hou, Xiang-Yang Li,
• Abstract summary: We propose a vision-based relation detection model, named ViRED, to identify the associations between tables and circuits in electrical engineering drawings. Within the engineering drawing dataset, our approach attained an accuracy of 96% in the task of relation prediction. Results also show that ViRED can inference at a fast speed even when there are numerous objects in a single engineering drawing.
• Score: 12.49418314310749
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: To accurately understand engineering drawings, it is essential to establish the correspondence between images and their description tables within the drawings. Existing document understanding methods predominantly focus on text as the main modality, which is not suitable for documents containing substantial image information. In the field of visual relation detection, the structure of the task inherently limits its capacity to assess relationships among all entity pairs in the drawings. To address this issue, we propose a vision-based relation detection model, named ViRED, to identify the associations between tables and circuits in electrical engineering drawings. Our model mainly consists of three parts: a vision encoder, an object encoder, and a relation decoder. We implement ViRED using PyTorch to evaluate its performance. To validate the efficacy of ViRED, we conduct a series of experiments. The experimental results indicate that, within the engineering drawing dataset, our approach attained an accuracy of 96\% in the task of relation prediction, marking a substantial improvement over existing methodologies. The results also show that ViRED can inference at a fast speed even when there are numerous objects in a single engineering drawing.

Related papers

• Composing Object Relations and Attributes for Image-Text Matching [70.47747937665987]
  This work introduces a dual-encoder image-text matching model, leveraging a scene graph to represent captions with nodes for objects and attributes interconnected by relational edges. Our model efficiently encodes object-attribute and object-object semantic relations, resulting in a robust and fast-performing system.
  arXiv  Detail & Related papers  (2024-06-17T17:56:01Z)
• EGTR: Extracting Graph from Transformer for Scene Graph Generation [5.935927309154952]
  Scene Graph Generation (SGG) is a challenging task of detecting objects and predicting relationships between objects. We propose a lightweight one-stage SGG model that extracts the relation graph from the various relationships learned in the multi-head self-attention layers of the DETR decoder. We demonstrate the effectiveness and efficiency of our method for the Visual Genome and Open Image V6 datasets.
  arXiv  Detail & Related papers  (2024-04-02T16:20:02Z)
• Can AI Models Appreciate Document Aesthetics? An Exploration of Legibility and Layout Quality in Relation to Prediction Confidence [3.049887057143419]
  A well-designed document communicates not only through its words but also through its visual eloquence. Authors utilize aesthetic elements such as colors, fonts, graphics, and layouts to shape the perception of information. While state-of-the-art document AI models demonstrate the benefits of incorporating layout and image data, it remains unclear whether the nuances of document aesthetics are effectively captured.
  arXiv  Detail & Related papers  (2024-03-27T01:21:48Z)
• Scene-Graph ViT: End-to-End Open-Vocabulary Visual Relationship Detection [14.22646492640906]
  We propose a simple and highly efficient decoder-free architecture for open-vocabulary visual relationship detection. Our model consists of a Transformer-based image encoder that represents objects as tokens and models their relationships implicitly. Our approach achieves state-of-the-art relationship detection performance on Visual Genome and on the large-vocabulary GQA benchmark at real-time inference speeds.
  arXiv  Detail & Related papers  (2024-03-21T10:15:57Z)
• Visual Commonsense based Heterogeneous Graph Contrastive Learning [79.22206720896664]
  We propose a heterogeneous graph contrastive learning method to better finish the visual reasoning task. Our method is designed as a plug-and-play way, so that it can be quickly and easily combined with a wide range of representative methods.
  arXiv  Detail & Related papers  (2023-11-11T12:01:18Z)
• Detecting Objects with Context-Likelihood Graphs and Graph Refinement [45.70356990655389]
  The goal of this paper is to detect objects by exploiting their ins. Contrary to existing methods, which learn objects and relations separately, our key idea is to learn the object-relation distribution jointly. We propose a novel way of creating a graphical representation of an image from inter-object relations and initial class predictions, we call a context-likelihood graph. We then learn the joint with an energy-based modeling technique which allows a sample and refine the context-likelihood graph iteratively for a given image.
  arXiv  Detail & Related papers  (2022-12-23T15:27:21Z)
• One-shot Scene Graph Generation [130.57405850346836]
  We propose Multiple Structured Knowledge (Relational Knowledgesense Knowledge) for the one-shot scene graph generation task. Our method significantly outperforms existing state-of-the-art methods by a large margin.
  arXiv  Detail & Related papers  (2022-02-22T11:32:59Z)
• Learning Co-segmentation by Segment Swapping for Retrieval and Discovery [67.6609943904996]
  The goal of this work is to efficiently identify visually similar patterns from a pair of images. We generate synthetic training pairs by selecting object segments in an image and copy-pasting them into another image. We show our approach provides clear improvements for artwork details retrieval on the Brueghel dataset.
  arXiv  Detail & Related papers  (2021-10-29T16:51:16Z)
• Tensor Composition Net for Visual Relationship Prediction [115.14829858763399]
  We present a novel Composition Network (TCN) to predict visual relationships in images. The key idea of our TCN is to exploit the low rank property of the visual relationship tensor. We show our TCN's image-level visual relationship prediction provides a simple and efficient mechanism for relation-based image retrieval.
  arXiv  Detail & Related papers  (2020-12-10T06:27:20Z)
• ConsNet: Learning Consistency Graph for Zero-Shot Human-Object Interaction Detection [101.56529337489417]
  We consider the problem of Human-Object Interaction (HOI) Detection, which aims to locate and recognize HOI instances in the form of human, action, object> in images. We argue that multi-level consistencies among objects, actions and interactions are strong cues for generating semantic representations of rare or previously unseen HOIs. Our model takes visual features of candidate human-object pairs and word embeddings of HOI labels as inputs, maps them into visual-semantic joint embedding space and obtains detection results by measuring their similarities.
  arXiv  Detail & Related papers  (2020-08-14T09:11:18Z)
• Visual Relationship Detection using Scene Graphs: A Survey [1.3505077405741583]
  A Scene Graph is a technique to better represent a scene and the various relationships present in it. We present a detailed survey on the various techniques for scene graph generation, their efficacy to represent visual relationships and how it has been used to solve various downstream tasks.
  arXiv  Detail & Related papers  (2020-05-16T17:06:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.