Image-Pointcloud Fusion based Anomaly Detection using PD-REAL Dataset

• URL: http://arxiv.org/abs/2311.04095v1
• Date: Tue, 7 Nov 2023 16:05:27 GMT
• Title: Image-Pointcloud Fusion based Anomaly Detection using PD-REAL Dataset
• Authors: Jianjian Qin, Chunzhi Gu, Jun Yu, Chao Zhang
• Abstract summary: We present PD-REAL, a novel large-scale dataset for unsupervised anomaly detection (AD) in the 3D domain. It is motivated by the fact that 2D-only representations in the AD task may fail to capture the geometric structures of anomalies due to uncertainty in lighting conditions or shooting angles. PD-REAL consists entirely of Play-Doh models for 15 object categories and focuses on the analysis of potential benefits from 3D information in a controlled environment.
• Score: 15.212031255539022
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present PD-REAL, a novel large-scale dataset for unsupervised anomaly detection (AD) in the 3D domain. It is motivated by the fact that 2D-only representations in the AD task may fail to capture the geometric structures of anomalies due to uncertainty in lighting conditions or shooting angles. PD-REAL consists entirely of Play-Doh models for 15 object categories and focuses on the analysis of potential benefits from 3D information in a controlled environment. Specifically, objects are first created with six types of anomalies, such as dent, crack, or perforation, and then photographed under different lighting conditions to mimic real-world inspection scenarios. To demonstrate the usefulness of 3D information, we use a commercially available RealSense camera to capture RGB and depth images. Compared to the existing 3D dataset for AD tasks, the data acquisition of PD-REAL is significantly cheaper, easily scalable and easier to control variables. Extensive evaluations with state-of-the-art AD algorithms on our dataset demonstrate the benefits as well as challenges of using 3D information. Our dataset can be downloaded from https://github.com/Andy-cs008/PD-REAL

Related papers

• DriveGEN: Generalized and Robust 3D Detection in Driving via Controllable Text-to-Image Diffusion Generation [49.32104127246474]
  DriveGEN is a training-free controllable Text-to-Image Diffusion Generation. It consistently preserves objects with precise 3D geometry across diverse Out-of-Distribution generations.
  arXiv  Detail & Related papers  (2025-03-14T06:35:38Z)
• PointAD: Comprehending 3D Anomalies from Points and Pixels for Zero-shot 3D Anomaly Detection [13.60524473223155]
  This paper introduces PointAD, a novel approach that transfers the strong generalization capabilities of CLIP for recognizing 3D anomalies on unseen objects. PointAD renders 3D anomalies into multiple 2D renderings and projects them back into 3D space. Our model can directly integrate RGB information, further enhancing the understanding of 3D anomalies in a plug-and-play manner.
  arXiv  Detail & Related papers  (2024-10-01T01:40:22Z)
• VFMM3D: Releasing the Potential of Image by Vision Foundation Model for Monocular 3D Object Detection [80.62052650370416]
  monocular 3D object detection holds significant importance across various applications, including autonomous driving and robotics. In this paper, we present VFMM3D, an innovative framework that leverages the capabilities of Vision Foundation Models (VFMs) to accurately transform single-view images into LiDAR point cloud representations.
  arXiv  Detail & Related papers  (2024-04-15T03:12:12Z)
• 3DiffTection: 3D Object Detection with Geometry-Aware Diffusion Features [70.50665869806188]
  3DiffTection is a state-of-the-art method for 3D object detection from single images. We fine-tune a diffusion model to perform novel view synthesis conditioned on a single image. We further train the model on target data with detection supervision.
  arXiv  Detail & Related papers  (2023-11-07T23:46:41Z)
• Real3D-AD: A Dataset of Point Cloud Anomaly Detection [75.56719157477661]
  We introduce Real3D-AD, a challenging high-precision point cloud anomaly detection dataset. With 1,254 high-resolution 3D items from forty thousand to millions of points for each item, Real3D-AD is the largest dataset for high-precision 3D industrial anomaly detection. We present a comprehensive benchmark for Real3D-AD, revealing the absence of baseline methods for high-precision point cloud anomaly detection.
  arXiv  Detail & Related papers  (2023-09-23T00:43:38Z)
• Digital Twin Tracking Dataset (DTTD): A New RGB+Depth 3D Dataset for Longer-Range Object Tracking Applications [3.9776693020673677]
  Digital twin is a problem of augmenting real objects with their digital counterparts. A critical component in a good digital-twin system is real-time, accurate 3D object tracking. In this work, we create a novel RGB-D dataset, called Digital Twin Tracking dataset (DTTD)
  arXiv  Detail & Related papers  (2023-02-12T20:06:07Z)
• Bridged Transformer for Vision and Point Cloud 3D Object Detection [92.86856146086316]
  Bridged Transformer (BrT) is an end-to-end architecture for 3D object detection. BrT learns to identify 3D and 2D object bounding boxes from both points and image patches. We experimentally show that BrT surpasses state-of-the-art methods on SUN RGB-D and ScanNetV2 datasets.
  arXiv  Detail & Related papers  (2022-10-04T05:44:22Z)
• 3D-VField: Learning to Adversarially Deform Point Clouds for Robust 3D Object Detection [111.32054128362427]
  In safety-critical settings, robustness on out-of-distribution and long-tail samples is fundamental to circumvent dangerous issues. We substantially improve the generalization of 3D object detectors to out-of-domain data by taking into account deformed point clouds during training. We propose and share open source CrashD: a synthetic dataset of realistic damaged and rare cars.
  arXiv  Detail & Related papers  (2021-12-09T08:50:54Z)
• SF-UDA$^{3D}$: Source-Free Unsupervised Domain Adaptation for LiDAR-Based 3D Object Detection [66.63707940938012]
  3D object detectors based only on LiDAR point clouds hold the state-of-the-art on modern street-view benchmarks. This paper proposes SF-UDA$3D$ to domain-adapt the state-of-the-art PointRCNN 3D detector to target domains for which we have no annotations.
  arXiv  Detail & Related papers  (2020-10-16T08:44:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.