A Synthetic-Driven Vision System for Assembly Step Recognition

• URL: http://arxiv.org/abs/2607.00129v1
• Date: Tue, 30 Jun 2026 20:08:47 GMT
• Title: A Synthetic-Driven Vision System for Assembly Step Recognition
• Abstract summary: We propose a system that automatically generates realistic assembly sequences and further trains real-time inspection models.<n>It can be efficiently applied to a given task within an hour, requiring only CAD models and simple step descriptions.<n>Overall, our system provides a practical solution for industrial assembly inspection without requiring expensive real-world data collection and annotation.
• Score: 5.6031705142358845
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Quality control in industrial assembly is essential, and real-time monitoring of the assembly process is crucial for preventing costly defects and ensuring production reliability. Vision-based automated inspection offers a powerful solution for such real-time monitoring. However, due to the specialized industrial components and processes, training these models typically relies on task-specific real-world data, which is costly and labor-intensive to collect and annotate. In this paper, we propose a system that automatically generates realistic assembly sequences and further trains real-time inspection models using the synthetic data. It can be efficiently applied to a given task within an hour, requiring only CAD models and simple step descriptions. Focusing on practical challenges, our system integrates a physics-based motion generation module to capture the variance of different human assembly, designs domain-randomized rendering to deal with the environmental complexity and variation, and employs an object-detection-based step recognition module for robust sim-to-real transfer, leading to 92.4% accuracy on a real-world assembly case with 46.7%, 15.8% and 61.2% performance improvement, respectively. Overall, our system provides a practical solution for industrial assembly inspection without requiring expensive real-world data collection and annotation, with the effectiveness validated on real industrial assembly tasks.

Related papers

• IPM-FM: A Foundation Model with Consensus Feature Selection for Industrial Process Monitoring [59.59850511287819]
  We propose the industrial process monitoring foundation model (IPM-FM)<n>It first learns general-purpose representations from unlabeled industrial process data through self-supervised pretraining.<n>It adapts to specific monitoring tasks using a small amount of task-labeled data, and finally produces calibrated predictions.
  arXiv  Detail & Related papers  (2026-09-08T07:43:04Z)
• MsFormer: Enabling Robust Predictive Maintenance Services for Industrial Devices [53.94035827454474]
  We propose a lightweight Multi-scale Transformer designed as a unified AI service model for reliable industrial predictive maintenance.<n>MsFormer incorporates a Multi-scale Sampling (MS) module and a tailored position encoding mechanism to capture sequential correlations.<n>Experiments on real-world datasets demonstrate that the proposed framework achieves significant performance improvements over state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-03-24T11:12:50Z)
• Prompt-Driven Lightweight Foundation Model for Instance Segmentation-Based Fault Detection in Freight Trains [6.6492664858930475]
  We propose a lightweight self-prompted instance segmentation framework tailored for freight train fault detection.<n>Our method leverages the Segment Anything Model by introducing a self-prompt generation module that automatically produces task-specific prompts.<n>In addition, we adopt a Tiny Vision Transformer backbone to reduce computational cost, making the framework suitable for real-time deployment on edge devices in railway monitoring systems.
  arXiv  Detail & Related papers  (2026-03-13T03:56:37Z)
• Zero-Shot Multi-Criteria Visual Quality Inspection for Semi-Controlled Industrial Environments via Real-Time 3D Digital Twin Simulation [5.0268543063681195]
  We propose a pose-agnostic, zero-shot quality inspection framework that compares real scenes against real-time Digital Twins (DT) in the RGB-D space.<n>Our approach enables efficient real-time DT rendering by semantically describing industrial scenes through object detection and pose estimation.<n>Based on an automotive use case featuring the quality inspection of an axial flux motor, we demonstrate the effectiveness of our framework.
  arXiv  Detail & Related papers  (2025-11-28T14:19:31Z)
• A Synthetic Data Pipeline for Supporting Manufacturing SMEs in Visual Assembly Control [0.0]
  We present a novel approach for easily integrable and data-efficient visual assembly control.<n>Our approach leverages simulated scene generation based on computer-aided design (CAD) data and object detection algorithms.<n>The results demonstrate a time-saving pipeline for generating image data in manufacturing environments.
  arXiv  Detail & Related papers  (2025-09-16T13:48:55Z)
• Learning-based Stage Verification System in Manual Assembly Scenarios [2.517043342442487]
  This study presents a novel approach to achieve precise monitoring under the limitation of using a minimal number of visual sensors.<n>By integrating state information from identical timestamps, our method detects and confirms the current stage of the assembly process with an average accuracy exceeding 92%.
  arXiv  Detail & Related papers  (2025-07-23T08:10:27Z)
• Evaluating Large Language Models for Real-World Engineering Tasks [75.97299249823972]
  This paper introduces a curated database comprising over 100 questions derived from authentic, production-oriented engineering scenarios.<n>Using this dataset, we evaluate four state-of-the-art Large Language Models (LLMs)<n>Our results show that LLMs demonstrate strengths in basic temporal and structural reasoning but struggle significantly with abstract reasoning, formal modeling, and context-sensitive engineering logic.
  arXiv  Detail & Related papers  (2025-05-12T14:05:23Z)
• AIDE: An Automatic Data Engine for Object Detection in Autonomous Driving [68.73885845181242]
  We propose an Automatic Data Engine (AIDE) that automatically identifies issues, efficiently curates data, improves the model through auto-labeling, and verifies the model through generation of diverse scenarios. We further establish a benchmark for open-world detection on AV datasets to comprehensively evaluate various learning paradigms, demonstrating our method's superior performance at a reduced cost.
  arXiv  Detail & Related papers  (2024-03-26T04:27:56Z)
• MMRNet: Improving Reliability for Multimodal Object Detection and Segmentation for Bin Picking via Multimodal Redundancy [68.7563053122698]
  We propose a reliable object detection and segmentation system with MultiModal Redundancy (MMRNet) This is the first system that introduces the concept of multimodal redundancy to address sensor failure issues during deployment. We present a new label-free multi-modal consistency (MC) score that utilizes the output from all modalities to measure the overall system output reliability and uncertainty.
  arXiv  Detail & Related papers  (2022-10-19T19:15:07Z)
• Scalable Vehicle Re-Identification via Self-Supervision [66.2562538902156]
  Vehicle Re-Identification is one of the key elements in city-scale vehicle analytics systems. Many state-of-the-art solutions for vehicle re-id mostly focus on improving the accuracy on existing re-id benchmarks and often ignore computational complexity. We propose a simple yet effective hybrid solution empowered by self-supervised training which only uses a single network during inference time.
  arXiv  Detail & Related papers  (2022-05-16T12:14:42Z)
• Industrial object, machine part and defect recognition towards fully automated industrial monitoring employing deep learning. The case of multilevel VGG19 [0.0]
  Modern industry requires modern solutions for monitoring the automatic production of goods. We propose a modified version of the Virtual Geometry Group (VGG) network, called Multipath VGG19, which allows for more local and global feature extraction. Specifically, top classification performance was achieved in five of the six image datasets, while the average classification improvement was 6.95%.
  arXiv  Detail & Related papers  (2020-11-23T10:05:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.