Dual-Branch Reconstruction Network for Industrial Anomaly Detection with RGB-D Data

• URL: http://arxiv.org/abs/2311.06797v1
• Date: Sun, 12 Nov 2023 10:19:14 GMT
• Title: Dual-Branch Reconstruction Network for Industrial Anomaly Detection with RGB-D Data
• Authors: Chenyang Bi, Yueyang Li and Haichi Luo
• Abstract summary: Multi-modal industrial anomaly detection based on 3D point clouds and RGB images is just beginning to emerge. The above methods require a longer inference time and higher memory usage, which cannot meet the real-time requirements of the industry. We propose a lightweight dual-branch reconstruction network based on RGB-D input, learning the decision boundary between normal and abnormal examples.
• Score: 1.861332908680942
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Unsupervised anomaly detection methods are at the forefront of industrial anomaly detection efforts and have made notable progress. Previous work primarily used 2D information as input, but multi-modal industrial anomaly detection based on 3D point clouds and RGB images is just beginning to emerge. The regular approach involves utilizing large pre-trained models for feature representation and storing them in memory banks. However, the above methods require a longer inference time and higher memory usage, which cannot meet the real-time requirements of the industry. To overcome these issues, we propose a lightweight dual-branch reconstruction network(DBRN) based on RGB-D input, learning the decision boundary between normal and abnormal examples. The requirement for alignment between the two modalities is eliminated by using depth maps instead of point cloud input. Furthermore, we introduce an importance scoring module in the discriminative network to assist in fusing features from these two modalities, thereby obtaining a comprehensive discriminative result. DBRN achieves 92.8% AUROC with high inference efficiency on the MVTec 3D-AD dataset without large pre-trained models and memory banks.

Related papers

• Memoryless Multimodal Anomaly Detection via Student-Teacher Network and Signed Distance Learning [8.610387986933741]
  A novel memoryless method MDSS is proposed for multimodal anomaly detection. It employs a light-weighted student-teacher network and a signed distance function to learn from RGB images and 3D point clouds respectively. The experimental results indicate that MDSS is comparable but more stable than the SOTA memory bank based method Shape-guided.
  arXiv  Detail & Related papers  (2024-09-09T07:18:09Z)
• M3DM-NR: RGB-3D Noisy-Resistant Industrial Anomaly Detection via Multimodal Denoising [63.39134873744748]
  Existing industrial anomaly detection methods primarily concentrate on unsupervised learning with pristine RGB images. This paper proposes a novel noise-resistant M3DM-NR framework to leverage strong multi-modal discriminative capabilities of CLIP. Extensive experiments show that M3DM-NR outperforms state-of-the-art methods in 3D-RGB multi-modal noisy anomaly detection.
  arXiv  Detail & Related papers  (2024-06-04T12:33:02Z)
• Self-supervised Feature Adaptation for 3D Industrial Anomaly Detection [59.41026558455904]
  We focus on multi-modal anomaly detection. Specifically, we investigate early multi-modal approaches that attempted to utilize models pre-trained on large-scale visual datasets. We propose a Local-to-global Self-supervised Feature Adaptation (LSFA) method to finetune the adaptors and learn task-oriented representation toward anomaly detection.
  arXiv  Detail & Related papers  (2024-01-06T07:30:41Z)
• PointMBF: A Multi-scale Bidirectional Fusion Network for Unsupervised RGB-D Point Cloud Registration [6.030097207369754]
  We propose a network implementing multi-scale bidirectional fusion between RGB images and point clouds generated from depth images. Our method achieves new state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-08-09T08:13:46Z)
• EasyNet: An Easy Network for 3D Industrial Anomaly Detection [49.26348455493123]
  3D anomaly detection is an emerging and vital computer vision task in industrial manufacturing. We propose an easy and deployment-friendly network (called EasyNet) without using pre-trained models and memory banks. Experiments show that EasyNet achieves an anomaly detection AUROC of 92.6% without using pre-trained models and memory banks.
  arXiv  Detail & Related papers  (2023-07-26T02:46:50Z)
• Multimodal Industrial Anomaly Detection via Hybrid Fusion [59.16333340582885]
  We propose a novel multimodal anomaly detection method with hybrid fusion scheme. Our model outperforms the state-of-the-art (SOTA) methods on both detection and segmentation precision on MVTecD-3 AD dataset.
  arXiv  Detail & Related papers  (2023-03-01T15:48:27Z)
• Self-Supervised Representation Learning for RGB-D Salient Object Detection [93.17479956795862]
  We use Self-Supervised Representation Learning to design two pretext tasks: the cross-modal auto-encoder and the depth-contour estimation. Our pretext tasks require only a few and un RGB-D datasets to perform pre-training, which make the network capture rich semantic contexts. For the inherent problem of cross-modal fusion in RGB-D SOD, we propose a multi-path fusion module.
  arXiv  Detail & Related papers  (2021-01-29T09:16:06Z)
• Learning Selective Mutual Attention and Contrast for RGB-D Saliency Detection [145.4919781325014]
  How to effectively fuse cross-modal information is the key problem for RGB-D salient object detection. Many models use the feature fusion strategy but are limited by the low-order point-to-point fusion methods. We propose a novel mutual attention model by fusing attention and contexts from different modalities.
  arXiv  Detail & Related papers  (2020-10-12T08:50:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.