Geometric 2D Scene Graph Generation

• URL: http://arxiv.org/abs/2607.22325v1
• Date: Fri, 24 Jul 2026 14:03:24 GMT
• Title: Geometric 2D Scene Graph Generation
• Abstract summary: We propose a method for constructing scene graphs to represent and characterize assembly relationships.<n>Our approach does not rely on semantic data and is capable of handling a very small dataset.<n>We validate our method on a study dataset of toy model components which can be assembled into transportation vehicles.
• Score: 4.798313004706817
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In production processes for consumer products, assembly instructions are essential not only for planning but also for executing the production process. Likewise in robotics, it is crucial for an assembly robot to understand how components fit together and can be assembled. To facilitate these tasks, we contribute a method for constructing scene graphs to represent and characterize assembly relationships between components. Our approach does not rely on semantic data and is capable of handling a very small dataset. To realize this, the output of a Faster R-CNN model is used to create geometric representations, which are then processed by a transformer architecture to generate an adjacency matrix. This matrix serves as input to a Siamese network that uses message passing based on an attentional graph convolutional network (aGCN) architecture to characterize the connections between the components. We validate our method on a study dataset of toy model components which can be assembled into transportation vehicles.

Related papers

• Iterative Compositional Data Generation for Robot Control [8.888011063034517]
  We propose a semantic compositional diffusion transformer that factorizes transitions into robot-, object-, obstacle-, and objective-specific components.<n>We show that our model can zero-shot generate high-quality transitions from which we can learn control policies for unseen task combinations.
  arXiv  Detail & Related papers  (2025-12-11T18:20:49Z)
• 3D Geometric Shape Assembly via Efficient Point Cloud Matching [59.241448711254485]
  We introduce Proxy Match Transform (PMT), an approximate high-order feature transform layer that enables reliable matching between mating surfaces of parts. Building upon PMT, we introduce a new framework, dubbed Proxy Match TransformeR (PMTR), for the geometric assembly task. We evaluate the proposed PMTR on the large-scale 3D geometric shape assembly benchmark dataset of Breaking Bad.
  arXiv  Detail & Related papers  (2024-07-15T08:50:02Z)
• Zero-shot Composed Text-Image Retrieval [72.43790281036584]
  We consider the problem of composed image retrieval (CIR) It aims to train a model that can fuse multi-modal information, e.g., text and images, to accurately retrieve images that match the query, extending the user's expression ability.
  arXiv  Detail & Related papers  (2023-06-12T17:56:01Z)
• Efficient and Feasible Robotic Assembly Sequence Planning via Graph Representation Learning [22.447462847331312]
  We propose a holistic graphical approach including a graph representation called Assembly Graph for product assemblies. With GRACE, we are able to extract meaningful information from the graph input and predict assembly sequences in a step-by-step manner. In experiments, we show that our approach can predict feasible assembly sequences across product variants of aluminum profiles.
  arXiv  Detail & Related papers  (2023-03-17T17:23:14Z)
• Component Segmentation of Engineering Drawings Using Graph Convolutional Networks [0.8941624592392744]
  We present a data-driven framework to automate the vectorization and machine interpretation of 2D engineering part drawings. To overcome these challenges, we propose a deep learning based framework that predicts the semantic type of each vectorized component. Results show that our method yields the best performance compared to recent image, and graph-based segmentation methods.
  arXiv  Detail & Related papers  (2022-12-01T05:31:07Z)
• 3D Part Assembly Generation with Instance Encoded Transformer [22.330218525999857]
  We propose a multi-layer transformer-based framework that involves geometric and relational reasoning between parts to update the part poses iteratively. We extend our framework to a new task called in-process part assembly. Our method achieves far more than 10% improvements over the current state-of-the-art in multiple metrics on the public PartNet dataset.
  arXiv  Detail & Related papers  (2022-07-05T02:40:57Z)
• RICE: Refining Instance Masks in Cluttered Environments with Graph Neural Networks [53.15260967235835]
  We propose a novel framework that refines the output of such methods by utilizing a graph-based representation of instance masks. We train deep networks capable of sampling smart perturbations to the segmentations, and a graph neural network, which can encode relations between objects, to evaluate the segmentations. We demonstrate an application that uses uncertainty estimates generated by our method to guide a manipulator, leading to efficient understanding of cluttered scenes.
  arXiv  Detail & Related papers  (2021-06-29T20:29:29Z)
• Graph Convolutional Embeddings for Recommender Systems [67.5973695167534]
  We propose a graph convolutional embedding layer for N-partite graphs that processes user-item-context interactions. More specifically, we define a graph convolutional embedding layer for N-partite graphs that processes user-item-context interactions.
  arXiv  Detail & Related papers  (2021-03-05T10:46:16Z)
• Where2Act: From Pixels to Actions for Articulated 3D Objects [54.19638599501286]
  We extract highly localized actionable information related to elementary actions such as pushing or pulling for articulated objects with movable parts. We propose a learning-from-interaction framework with an online data sampling strategy that allows us to train the network in simulation. Our learned models even transfer to real-world data.
  arXiv  Detail & Related papers  (2021-01-07T18:56:38Z)
• Quiver Signal Processing (QSP) [145.6921439353007]
  We state the basics for a signal processing framework on quiver representations. We propose a signal processing framework that allows us to handle heterogeneous multidimensional information in networks.
  arXiv  Detail & Related papers  (2020-10-22T08:40:15Z)
• Generative 3D Part Assembly via Dynamic Graph Learning [34.108515032411695]
  Part assembly is a challenging yet crucial task in 3D computer vision and robotics. We propose an assembly-oriented dynamic graph learning framework that leverages an iterative graph neural network as a backbone.
  arXiv  Detail & Related papers  (2020-06-14T04:26:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.