Towards Open-Vocabulary Industrial Defect Understanding with a Large-Scale Multimodal Dataset

• URL: http://arxiv.org/abs/2512.24160v1
• Date: Tue, 30 Dec 2025 11:45:22 GMT
• Title: Towards Open-Vocabulary Industrial Defect Understanding with a Large-Scale Multimodal Dataset
• Abstract summary: We present IMDD-1M, the first large-scale Industrial Multimodal Defect dataset comprising 1,000,000 aligned image-text pairs.<n>This dataset enables a wide spectrum of applications, including classification, segmentation, retrieval, captioning, and generative modeling.
• Score: 1.1470070927586018
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present IMDD-1M, the first large-scale Industrial Multimodal Defect Dataset comprising 1,000,000 aligned image-text pairs, designed to advance multimodal learning for manufacturing and quality inspection. IMDD-1M contains high-resolution real-world defects spanning over 60 material categories and more than 400 defect types, each accompanied by expert-verified annotations and fine-grained textual descriptions detailing defect location, severity, and contextual attributes. This dataset enables a wide spectrum of applications, including classification, segmentation, retrieval, captioning, and generative modeling. Building upon IMDD-1M, we train a diffusion-based vision-language foundation model from scratch, specifically tailored for industrial scenarios. The model serves as a generalizable foundation that can be efficiently adapted to specialized domains through lightweight fine-tuning. With less than 5% of the task-specific data required by dedicated expert models, it achieves comparable performance, highlighting the potential of data-efficient foundation model adaptation for industrial inspection and generation, paving the way for scalable, domain-adaptive, and knowledge-grounded manufacturing intelligence.

Related papers

• Unification of Closed-Open Industrial Detection Scenarios: New Large-Scale Benchmarks,Challenges and Baselines [50.471211036005286]
  We introduce a Large-Scale Multi-Modal Industrial Open-Closed benchmark (MMIOC-1M)<n> MMIOC-1M is the first unified largest benchmark supporting both open-vocabulary and closed-set industrial detection.<n>We propose a Refined Text-Visual Prompt Network (RTVPNet) that incorporates three key innovations.
  arXiv  Detail & Related papers  (2026-06-06T03:06:10Z)
• FORGE: Fine-grained Multimodal Evaluation for Manufacturing Scenarios [58.34124792457706]
  The manufacturing sector is increasingly adopting Multimodal Large Language Models (MLLMs) to transition from simple perception to autonomous execution.<n>Progress is hindered by data scarcity and a lack of fine-grained domain semantics in existing datasets.<n>We first construct a high-quality dataset that combines realworld 2D images and 3D point clouds, annotated with fine-grained domain semantics.<n>We then evaluate 18 state-of-the-art MLLMs across three manufacturing tasks, namely verification, structural surface inspection, and assembly verification, revealing significant performance gaps.
  arXiv  Detail & Related papers  (2026-04-08T12:23:27Z)
• Harnessing the Power of Foundation Models for Accurate Material Classification [0.22940141855172033]
  Material classification has emerged as a critical task in computer vision and graphics.<n>Recent advances in vision-language foundation models offer promising avenues to address these issues.<n>We propose a novel framework that effectively harnesses foundation models to overcome data limitations and enhance classification accuracy.
  arXiv  Detail & Related papers  (2026-03-18T06:14:00Z)
• Domain Specific Specialization in Low-Resource Settings: The Efficacy of Offline Response-Based Knowledge Distillation in Large Language Models [0.0]
  Large Language Models (LLMs) excel in general tasks but often struggle with hallucinations when handling domain-specific or institutional knowledge absent from pre-training.<n>We present an offline response-based knowledge distillation method that develops high-accuracy specialized assistants under constrained hardware resources.
  arXiv  Detail & Related papers  (2026-01-05T18:00:16Z)
• VITAL: Vision-Encoder-centered Pre-training for LMMs in Visual Quality Assessment [88.83260031198023]
  We propose a vision-encoder-centered generative pre-training pipeline and develop the VITAL-Series LMMs.<n>We construct over 4.5M vision-language (VL) pairs-the largest VQualA training dataset to date.<n>We employ a multi-task training workflow that simultaneously enhances the model's quantitative scoring precision.
  arXiv  Detail & Related papers  (2025-11-22T07:55:21Z)
• Real-IAD Variety: Pushing Industrial Anomaly Detection Dataset to a Modern Era [110.83702639978469]
  Real-IAD Variety is the largest and most diverse IAD benchmark, comprising 198,960 high-resolution images across 160 distinct object categories.<n>Its diversity is ensured through comprehensive coverage of 28 industries, 24 material types, and 22 color variations.<n>Real-IAD Variety will be made publicly available to facilitate innovation in this critical field.
  arXiv  Detail & Related papers  (2025-11-01T12:58:02Z)
• SoM-1K: A Thousand-Problem Benchmark Dataset for Strength of Materials [16.756001896133757]
  We introduce SoM-1K, the first large-scale multimodal benchmark dataset dedicated to evaluating foundation models on problems in the strength of materials.<n>This work establishes a rigorous benchmark for engineering AI and highlights a critical need for developing more robust multimodal reasoning capabilities.
  arXiv  Detail & Related papers  (2025-09-25T12:28:22Z)
• LimiX: Unleashing Structured-Data Modeling Capability for Generalist Intelligence [61.46575527504109]
  LimiX-16M and LimiX-2M treat structured data as a joint distribution over variables and missingness.<n>We evaluate LimiX models across 11 large structured-data benchmarks with broad regimes of sample size, feature dimensionality, class number, categorical-to-numerical feature ratio, missingness, and sample-to-feature ratios.
  arXiv  Detail & Related papers  (2025-09-03T17:39:08Z)
• Vision-G1: Towards General Vision Language Reasoning with Multi-Domain Data Curation [64.23194519770897]
  We build a comprehensive RL-ready visual reasoning dataset from 46 data sources across 8 dimensions.<n>We propose an influence function based data selection and difficulty based filtering strategy to identify high-quality training samples from this dataset.<n>We train the VLM, referred to as Vision-G1, using multi-round RL with a data curriculum to iteratively improve its visual reasoning capabilities.
  arXiv  Detail & Related papers  (2025-08-18T07:24:33Z)
• EIAD: Explainable Industrial Anomaly Detection Via Multi-Modal Large Language Models [23.898938659720503]
  Industrial Anomaly Detection (IAD) is critical to ensure product quality during manufacturing.<n>We propose a novel approach that introduces a dedicated multi-modal defect localization module to decouple the dialog functionality from the core feature extraction.<n>We also contribute to the first multi-modal industrial anomaly detection training dataset, named Defect Detection Question Answering (DDQA)
  arXiv  Detail & Related papers  (2025-03-18T11:33:29Z)
• mmE5: Improving Multimodal Multilingual Embeddings via High-quality Synthetic Data [71.352883755806]
  Multimodal embedding models have gained significant attention for their ability to map data from different modalities, such as text and images, into a unified representation space.<n>However, the limited labeled multimodal data often hinders embedding performance.<n>Recent approaches have leveraged data synthesis to address this problem, yet the quality of synthetic data remains a critical bottleneck.
  arXiv  Detail & Related papers  (2025-02-12T15:03:33Z)
• OReole-FM: successes and challenges toward billion-parameter foundation models for high-resolution satellite imagery [0.3926357402982764]
  Scaling models to billions of parameters has been shown to yield unprecedented benefits including emergent abilities. We pair high-performance computing resources including Frontier supercomputer, America's first exascale system, and high-resolution optical RS data to pretrain billion-scale FMs.
  arXiv  Detail & Related papers  (2024-10-25T20:55:12Z)
• NVLM: Open Frontier-Class Multimodal LLMs [64.00053046838225]
  We introduce NVLM 1.0, a family of frontier-class multimodal large language models (LLMs) that achieve state-of-the-art results on vision-language tasks. We propose a novel architecture that enhances both training efficiency and multimodal reasoning capabilities. We develop production-grade multimodality for the NVLM-1.0 models, enabling them to excel in vision-language tasks.
  arXiv  Detail & Related papers  (2024-09-17T17:59:06Z)
• Dial-insight: Fine-tuning Large Language Models with High-Quality Domain-Specific Data Preventing Capability Collapse [4.98050508891467]
  We propose a two-stage approach for the construction of production prompts designed to yield high-quality data. This method involves the generation of a diverse array of prompts that encompass a broad spectrum of tasks and exhibit a rich variety of expressions. We introduce a cost-effective, multi-dimensional quality assessment framework to ensure the integrity of the generated labeling data.
  arXiv  Detail & Related papers  (2024-03-14T08:27:32Z)
• Myriad: Large Multimodal Model by Applying Vision Experts for Industrial Anomaly Detection [86.24898024621008]
  We present a novel large multimodal model applying vision experts for industrial anomaly detection(abbreviated to Myriad)<n>We utilize the anomaly map generated by the vision experts as guidance for LMMs, such that the vision model is guided to pay more attention to anomalous regions.<n>Our proposed method not only performs favorably against state-of-the-art methods, but also inherits the flexibility and instruction-following ability of LMMs in the field of IAD.
  arXiv  Detail & Related papers  (2023-10-29T16:49:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.