Zero-Shot Multi-Criteria Visual Quality Inspection for Semi-Controlled Industrial Environments via Real-Time 3D Digital Twin Simulation

• URL: http://arxiv.org/abs/2511.23214v1
• Date: Fri, 28 Nov 2025 14:19:31 GMT
• Title: Zero-Shot Multi-Criteria Visual Quality Inspection for Semi-Controlled Industrial Environments via Real-Time 3D Digital Twin Simulation
• Authors: Jose Moises Araya-Martinez, Gautham Mohan, Kenichi Hayakawa Bolaños, Roberto Mendieta, Sarvenaz Sardari, Jens Lambrecht, Jörg Krüger,
• Abstract summary: We propose a pose-agnostic, zero-shot quality inspection framework that compares real scenes against real-time Digital Twins (DT) in the RGB-D space.<n>Our approach enables efficient real-time DT rendering by semantically describing industrial scenes through object detection and pose estimation.<n>Based on an automotive use case featuring the quality inspection of an axial flux motor, we demonstrate the effectiveness of our framework.
• Score: 5.0268543063681195
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Early-stage visual quality inspection is vital for achieving Zero-Defect Manufacturing and minimizing production waste in modern industrial environments. However, the complexity of robust visual inspection systems and their extensive data requirements hinder widespread adoption in semi-controlled industrial settings. In this context, we propose a pose-agnostic, zero-shot quality inspection framework that compares real scenes against real-time Digital Twins (DT) in the RGB-D space. Our approach enables efficient real-time DT rendering by semantically describing industrial scenes through object detection and pose estimation of known Computer-Aided Design models. We benchmark tools for real-time, multimodal RGB-D DT creation while tracking consumption of computational resources. Additionally, we provide an extensible and hierarchical annotation strategy for multi-criteria defect detection, unifying pose labelling with logical and structural defect annotations. Based on an automotive use case featuring the quality inspection of an axial flux motor, we demonstrate the effectiveness of our framework. Our results demonstrate detection performace, achieving intersection-over-union (IoU) scores of up to 63.3% compared to ground-truth masks, even if using simple distance measurements under semi-controlled industrial conditions. Our findings lay the groundwork for future research on generalizable, low-data defect detection methods in dynamic manufacturing settings.

Related papers

• 3D-ADAM: A Dataset for 3D Anomaly Detection in Additive Manufacturing [8.279480828330183]
  3D-ADAM is the first large-scale, industry-relevant dataset for RGB+3D surface defect detection in additive manufacturing.<n>3D-ADAM comprises 14,120 high-resolution scans of 217 unique parts, captured with four industrial depth sensors.<n>It includes 27,346 annotated defects across 12 categories along with 27,346 annotations of machine element features in 16 classes.
  arXiv  Detail & Related papers  (2025-07-10T15:09:20Z)
• Bounding Box-Guided Diffusion for Synthesizing Industrial Images and Segmentation Map [50.21082069320818]
  We propose a novel diffusion-based pipeline for generating high-fidelity industrial datasets with minimal supervision.<n>Our approach conditions the diffusion model on enriched bounding box representations to produce precise segmentation masks.<n>Results demonstrate that diffusion-based synthesis can bridge the gap between artificial and real-world industrial data.
  arXiv  Detail & Related papers  (2025-05-06T15:21:36Z)
• SteelBlastQC: Shot-blasted Steel Surface Dataset with Interpretable Detection of Surface Defects [0.04783917893588482]
  This study presents a dataset of 1654 labeled RGB images (512x512) of steel surfaces, classified as either "ready for paint" or "needs shot-blasting"<n>The dataset captures real-world surface defects, including discoloration, welding lines, scratches and corrosion.<n>We present interpretable decision-making by all three neural networks, allowing industry users to visually pinpoint problematic regions.
  arXiv  Detail & Related papers  (2025-04-29T07:51:58Z)
• ISP-AD: A Large-Scale Real-World Dataset for Advancing Industrial Anomaly Detection with Synthetic and Real Defects [0.0]
  TheISP-AD is the largest publicly available industrial dataset to date, including both synthetic and real defects collected directly from the factory floor.<n>Experiments show that even a small amount of injected, weakly labeled real defects improves generalization.
  arXiv  Detail & Related papers  (2025-03-06T21:56:31Z)
• Automatic Prompt Generation and Grounding Object Detection for Zero-Shot Image Anomaly Detection [17.06832015516288]
  We propose a zero-shot training-free approach for automated industrial image anomaly detection using a multimodal machine learning pipeline.<n>Our proposed model enables efficient, scalable, and objective quality control in industrial manufacturing settings.
  arXiv  Detail & Related papers  (2024-11-28T15:42:32Z)
• Uncertainty Estimation for 3D Object Detection via Evidential Learning [63.61283174146648]
  We introduce a framework for quantifying uncertainty in 3D object detection by leveraging an evidential learning loss on Bird's Eye View representations in the 3D detector. We demonstrate both the efficacy and importance of these uncertainty estimates on identifying out-of-distribution scenes, poorly localized objects, and missing (false negative) detections.
  arXiv  Detail & Related papers  (2024-10-31T13:13:32Z)
• IPAD: Industrial Process Anomaly Detection Dataset [71.39058003212614]
  Video anomaly detection (VAD) is a challenging task aiming to recognize anomalies in video frames. We propose a new dataset, IPAD, specifically designed for VAD in industrial scenarios. This dataset covers 16 different industrial devices and contains over 6 hours of both synthetic and real-world video footage.
  arXiv  Detail & Related papers  (2024-04-23T13:38:01Z)
• Investigation of the Impact of Synthetic Training Data in the Industrial Application of Terminal Strip Object Detection [4.327763441385371]
  In this paper, we investigate the sim-to-real generalization performance of standard object detectors on the complex industrial application of terminal strip object detection. We manually annotated 300 real images of terminal strips for the evaluation. The results show the cruciality of the objects of interest to have the same scale in either domain.
  arXiv  Detail & Related papers  (2024-03-06T18:33:27Z)
• Self-supervised Feature Adaptation for 3D Industrial Anomaly Detection [59.41026558455904]
  We focus on multi-modal anomaly detection. Specifically, we investigate early multi-modal approaches that attempted to utilize models pre-trained on large-scale visual datasets. We propose a Local-to-global Self-supervised Feature Adaptation (LSFA) method to finetune the adaptors and learn task-oriented representation toward anomaly detection.
  arXiv  Detail & Related papers  (2024-01-06T07:30:41Z)
• Spatial-Temporal Graph Enhanced DETR Towards Multi-Frame 3D Object Detection [54.041049052843604]
  We present STEMD, a novel end-to-end framework that enhances the DETR-like paradigm for multi-frame 3D object detection. First, to model the inter-object spatial interaction and complex temporal dependencies, we introduce the spatial-temporal graph attention network. Finally, it poses a challenge for the network to distinguish between the positive query and other highly similar queries that are not the best match.
  arXiv  Detail & Related papers  (2023-07-01T13:53:14Z)
• Cycle and Semantic Consistent Adversarial Domain Adaptation for Reducing Simulation-to-Real Domain Shift in LiDAR Bird's Eye View [110.83289076967895]
  We present a BEV domain adaptation method based on CycleGAN that uses prior semantic classification in order to preserve the information of small objects of interest during the domain adaptation process. The quality of the generated BEVs has been evaluated using a state-of-the-art 3D object detection framework at KITTI 3D Object Detection Benchmark.
  arXiv  Detail & Related papers  (2021-04-22T12:47:37Z)
• Cognitive Visual Inspection Service for LCD Manufacturing Industry [80.63336968475889]
  This paper discloses a novel visual inspection system for liquid crystal display (LCD), which is currently a dominant type in the FPD industry. System is based on two cornerstones: robust/high-performance defect recognition model and cognitive visual inspection service architecture.
  arXiv  Detail & Related papers  (2021-01-11T08:14:35Z)
• Real-World Anomaly Detection by using Digital Twin Systems and Weakly-Supervised Learning [3.0100975935933567]
  We present novel weakly-supervised approaches to anomaly detection for industrial settings. The approaches make use of a Digital Twin to generate a training dataset which simulates the normal operation of the machinery. The performance of the proposed methods is compared against various state-of-the-art anomaly detection algorithms on an application to a real-world dataset.
  arXiv  Detail & Related papers  (2020-11-12T10:15:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.