AD-Copilot: A Vision-Language Assistant for Industrial Anomaly Detection via Visual In-context Comparison

• URL: http://arxiv.org/abs/2603.13779v1
• Date: Sat, 14 Mar 2026 06:14:44 GMT
• Title: AD-Copilot: A Vision-Language Assistant for Industrial Anomaly Detection via Visual In-context Comparison
• Abstract summary: We present AD-Copilot, an interactive MLLM specialized for industrial anomaly detection (IAD)<n>We first design a novel data pipeline to mine inspection knowledge from sparsely labeled industrial images.<n>We then generate precise samples for captioning, VQA, and defect localization, yielding a large-scale multimodal Comparison-AD rich in semantic signals for IAD.<n>Experiments show that AD-Copilot achieves 82.3% accuracy on the MMAD benchmark, outperforming all other models without any data leakage.
• Score: 89.0720931534819
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multimodal Large Language Models (MLLMs) have achieved impressive success in natural visual understanding, yet they consistently underperform in industrial anomaly detection (IAD). This is because MLLMs trained mostly on general web data differ significantly from industrial images. Moreover, they encode each image independently and can only compare images in the language space, making them insensitive to subtle visual differences that are key to IAD. To tackle these issues, we present AD-Copilot, an interactive MLLM specialized for IAD via visual in-context comparison. We first design a novel data curation pipeline to mine inspection knowledge from sparsely labeled industrial images and generate precise samples for captioning, VQA, and defect localization, yielding a large-scale multimodal dataset Chat-AD rich in semantic signals for IAD. On this foundation, AD-Copilot incorporates a novel Comparison Encoder that employs cross-attention between paired image features to enhance multi-image fine-grained perception, and is trained with a multi-stage strategy that incorporates domain knowledge and gradually enhances IAD skills. In addition, we introduce MMAD-BBox, an extended benchmark for anomaly localization with bounding-box-based evaluation. The experiments show that AD-Copilot achieves 82.3% accuracy on the MMAD benchmark, outperforming all other models without any data leakage. In the MMAD-BBox test, it achieves a maximum improvement of $3.35\times$ over the baseline. AD-Copilot also exhibits excellent generalization of its performance gains across other specialized and general-purpose benchmarks. Remarkably, AD-Copilot surpasses human expert-level performance on several IAD tasks, demonstrating its potential as a reliable assistant for real-world industrial inspection. All datasets and models will be released for the broader benefit of the community.

Related papers

• DiG: Differential Grounding for Enhancing Fine-Grained Perception in Multimodal Large Language Model [22.28268642142352]
  DiG (Differential Grounding) is a novel proxy task framework where MLLMs learn fine-grained perception by identifying and localizing all differences between similar image pairs without prior knowledge of their number.<n>Our results highlight differential grounding as a scalable and robust approach for advancing fine-grained visual reasoning in MLLMs.
  arXiv  Detail & Related papers  (2025-12-14T10:40:27Z)
• IAD-GPT: Advancing Visual Knowledge in Multimodal Large Language Model for Industrial Anomaly Detection [70.02774285130238]
  This paper explores the combination of rich text semantics with both image-level and pixel-level information from images.<n>We propose IAD-GPT, a novel paradigm based on MLLMs for Industrial Anomaly Detection.<n>Experiments on MVTec-AD and VisA datasets demonstrate our state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-10-16T02:48:05Z)
• IADGPT: Unified LVLM for Few-Shot Industrial Anomaly Detection, Localization, and Reasoning via In-Context Learning [18.078896149087576]
  Few-Shot Industrial Anomaly Detection (FS-IAD) has important applications in automating industrial quality inspection.<n>We propose a unified framework, IADGPT, designed to perform FS-IAD in a human-like manner.<n>We present a new dataset comprising 100K images across 400 diverse industrial product categories with extensive-level textual annotations.
  arXiv  Detail & Related papers  (2025-08-14T14:24:47Z)
• Real-IAD D3: A Real-World 2D/Pseudo-3D/3D Dataset for Industrial Anomaly Detection [53.2590751089607]
  Real-IAD D3 is a high-precision multimodal dataset that incorporates an additional pseudo3D modality generated through photometric stereo.<n>We introduce an effective approach that integrates RGB, point cloud, and pseudo-3D depth information to leverage the complementary strengths of each modality.<n>Our experiments highlight the importance of these modalities in boosting detection robustness and overall IAD performance.
  arXiv  Detail & Related papers  (2025-04-19T08:05:47Z)
• M3-AGIQA: Multimodal, Multi-Round, Multi-Aspect AI-Generated Image Quality Assessment [65.3860007085689]
  M3-AGIQA is a comprehensive framework that enables more human-aligned, holistic evaluation of AI-generated images.<n>By aligning model outputs more closely with human judgment, M3-AGIQA delivers robust and interpretable quality scores.
  arXiv  Detail & Related papers  (2025-02-21T03:05:45Z)
• Towards Zero-Shot Anomaly Detection and Reasoning with Multimodal Large Language Models [29.078437003042357]
  Zero-Shot Anomaly Detection (ZSAD) is an emerging AD paradigm.<n>We propose Anomaly-OneVision (Anomaly-OV), the first specialist visual assistant for ZSAD and reasoning.
  arXiv  Detail & Related papers  (2025-02-11T14:50:43Z)
• Learning Feature Inversion for Multi-class Anomaly Detection under General-purpose COCO-AD Benchmark [101.23684938489413]
  Anomaly detection (AD) is often focused on detecting anomalies for industrial quality inspection and medical lesion examination. This work first constructs a large-scale and general-purpose COCO-AD dataset by extending COCO to the AD field. Inspired by the metrics in the segmentation field, we propose several more practical threshold-dependent AD-specific metrics.
  arXiv  Detail & Related papers  (2024-04-16T17:38:26Z)
• Raising the Bar of AI-generated Image Detection with CLIP [50.345365081177555]
  The aim of this work is to explore the potential of pre-trained vision-language models (VLMs) for universal detection of AI-generated images. We develop a lightweight detection strategy based on CLIP features and study its performance in a wide variety of challenging scenarios.
  arXiv  Detail & Related papers  (2023-11-30T21:11:20Z)
• AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Models [30.723122000372538]
  AnomalyGPT is a novel IAD approach based on Large Vision-Language Models (LVLM) We generate training data by simulating anomalous images and producing corresponding textual descriptions for each image. AnomalyGPT achieves the state-of-the-art performance with an accuracy of 86.1%, an image-level AUC of 94.1%, and a pixel-level AUC of 95.3% on the MVTec-AD dataset.
  arXiv  Detail & Related papers  (2023-08-29T15:02:53Z)
• IM-IAD: Industrial Image Anomaly Detection Benchmark in Manufacturing [88.35145788575348]
  Image anomaly detection (IAD) is an emerging and vital computer vision task in industrial manufacturing. The lack of a uniform IM benchmark is hindering the development and usage of IAD methods in real-world applications. We construct a comprehensive image anomaly detection benchmark (IM-IAD), which includes 19 algorithms on seven major datasets.
  arXiv  Detail & Related papers  (2023-01-31T01:24:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.