Component Segmentation of Engineering Drawings Using Graph Convolutional Networks

• URL: http://arxiv.org/abs/2212.00290v1
• Date: Thu, 1 Dec 2022 05:31:07 GMT
• Title: Component Segmentation of Engineering Drawings Using Graph Convolutional Networks
• Authors: Wentai Zhang, Joe Joseph, Yue Yin, Liuyue Xie, Tomotake Furuhata, Soji Yamakawa, Kenji Shimada, Levent Burak Kara
• Abstract summary: We present a data-driven framework to automate the vectorization and machine interpretation of 2D engineering part drawings. To overcome these challenges, we propose a deep learning based framework that predicts the semantic type of each vectorized component. Results show that our method yields the best performance compared to recent image, and graph-based segmentation methods.
• Score: 0.8941624592392744
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a data-driven framework to automate the vectorization and machine interpretation of 2D engineering part drawings. In industrial settings, most manufacturing engineers still rely on manual reads to identify the topological and manufacturing requirements from drawings submitted by designers. The interpretation process is laborious and time-consuming, which severely inhibits the efficiency of part quotation and manufacturing tasks. While recent advances in image-based computer vision methods have demonstrated great potential in interpreting natural images through semantic segmentation approaches, the application of such methods in parsing engineering technical drawings into semantically accurate components remains a significant challenge. The severe pixel sparsity in engineering drawings also restricts the effective featurization of image-based data-driven methods. To overcome these challenges, we propose a deep learning based framework that predicts the semantic type of each vectorized component. Taking a raster image as input, we vectorize all components through thinning, stroke tracing, and cubic bezier fitting. Then a graph of such components is generated based on the connectivity between the components. Finally, a graph convolutional neural network is trained on this graph data to identify the semantic type of each component. We test our framework in the context of semantic segmentation of text, dimension and, contour components in engineering drawings. Results show that our method yields the best performance compared to recent image, and graph-based segmentation methods.

Related papers

• VectorGraphNET: Graph Attention Networks for Accurate Segmentation of Complex Technical Drawings [0.40964539027092917]
  This paper introduces a new approach to extract and analyze vector data from technical drawings in PDF format. Our method involves converting PDF files into SVG format and creating a feature-rich graph representation. We then apply a graph attention transformer with hierarchical label definition to achieve accurate line-level segmentation.
  arXiv  Detail & Related papers  (2024-10-02T08:53:20Z)
• Systematic review of image segmentation using complex networks [1.3053649021965603]
  This review presents various image segmentation methods using complex networks. In computer vision and image processing applications, image segmentation is essential for analyzing complex images.
  arXiv  Detail & Related papers  (2024-01-05T11:14:07Z)
• Coarse-to-Fine Contrastive Learning in Image-Text-Graph Space for Improved Vision-Language Compositionality [50.48859793121308]
  Contrastively trained vision-language models have achieved remarkable progress in vision and language representation learning. Recent research has highlighted severe limitations in their ability to perform compositional reasoning over objects, attributes, and relations.
  arXiv  Detail & Related papers  (2023-05-23T08:28:38Z)
• I Know What You Draw: Learning Grasp Detection Conditioned on a Few Freehand Sketches [74.63313641583602]
  We propose a method to generate a potential grasp configuration relevant to the sketch-depicted objects. Our model is trained and tested in an end-to-end manner which is easy to be implemented in real-world applications.
  arXiv  Detail & Related papers  (2022-05-09T04:23:36Z)
• Geometric Understanding of Sketches [0.0]
  I explore two methods that help a system provide a geometric machine-understanding of sketches, and in-turn help a user accomplish a downstream task. The first work deals with interpretation of a 2D-line drawing as a graph structure, and also illustrates its effectiveness through its physical reconstruction by a robot. In the second work, we test the 3D-geometric understanding of a sketch-based system without explicit access to the information about 3D-geometry.
  arXiv  Detail & Related papers  (2022-04-13T23:55:51Z)
• SGEITL: Scene Graph Enhanced Image-Text Learning for Visual Commonsense Reasoning [61.57887011165744]
  multimodal Transformers have made great progress in the task of Visual Commonsense Reasoning. We propose a Scene Graph Enhanced Image-Text Learning framework to incorporate visual scene graphs in commonsense reasoning.
  arXiv  Detail & Related papers  (2021-12-16T03:16:30Z)
• Learning to Segment Human Body Parts with Synthetically Trained Deep Convolutional Networks [58.0240970093372]
  This paper presents a new framework for human body part segmentation based on Deep Convolutional Neural Networks trained using only synthetic data. The proposed approach achieves cutting-edge results without the need of training the models with real annotated data of human body parts.
  arXiv  Detail & Related papers  (2021-02-02T12:26:50Z)
• Adaptive Graph Representation Learning and Reasoning for Face Parsing [55.086151726427104]
  Face parsing infers a pixel-wise label to each facial component. Component-wise relationship is a critical clue in discriminating ambiguous pixels in facial area. We propose adaptive graph representation learning and reasoning over facial components.
  arXiv  Detail & Related papers  (2021-01-18T12:17:40Z)
• Towards Efficient Scene Understanding via Squeeze Reasoning [71.1139549949694]
  We propose a novel framework called Squeeze Reasoning. Instead of propagating information on the spatial map, we first learn to squeeze the input feature into a channel-wise global vector. We show that our approach can be modularized as an end-to-end trained block and can be easily plugged into existing networks.
  arXiv  Detail & Related papers  (2020-11-06T12:17:01Z)
• A Survey on Deep Learning Methods for Semantic Image Segmentation in Real-Time [0.0]
  In many areas, such as robotics and autonomous vehicles, semantic image segmentation is crucial. The success of medical diagnosis and treatment relies on the extremely accurate understanding of the data under consideration. Recent developments in deep learning have provided a host of tools to tackle this problem efficiently and with increased accuracy.
  arXiv  Detail & Related papers  (2020-09-27T20:30:10Z)
• Contextual Hourglass Network for Semantic Segmentation of High Resolution Aerial Imagery [5.694721155544124]
  We develop a novel semantic segmentation method and call it Contextual Hourglass Network. In our method, in order to improve the robustness of the prediction, we design a new contextual hourglass module. We further exploit the stacked encoder-decoder structure by connecting multiple contextual hourglass modules.
  arXiv  Detail & Related papers  (2018-10-30T15:33:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.