Unification of Closed-Open Industrial Detection Scenarios: New Large-Scale Benchmarks,Challenges and Baselines

• URL: http://arxiv.org/abs/2606.07953v1
• Date: Sat, 06 Jun 2026 03:06:10 GMT
• Title: Unification of Closed-Open Industrial Detection Scenarios: New Large-Scale Benchmarks,Challenges and Baselines
• Abstract summary: We introduce a Large-Scale Multi-Modal Industrial Open-Closed benchmark (MMIOC-1M)<n> MMIOC-1M is the first unified largest benchmark supporting both open-vocabulary and closed-set industrial detection.<n>We propose a Refined Text-Visual Prompt Network (RTVPNet) that incorporates three key innovations.
• Score: 50.471211036005286
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large-scale Visual-Language Models (LVLMs) have achieved remarkable success in natural visual tasks, yet their application to industrial defect detection remains challenging due to two fundamental limitations: (i) the scarcity of large-scale industrial datasets that cover diverse defect categories across multiple domains, and (ii) the reliance on manual prompts (points, boxes, masks) that introduce subjective noise and lack text-visual interaction for fine-grained understanding. To address these challenges, we introduce a Large-Scale Multi-Modal Industrial Open-Closed benchmark (MMIOC-1M) containing over one million samples across $14$ super-categories, $29$ industrial scenes, and $351$ defect subcategories. To our knowledge, MMIOC-1M is the first unified largest benchmark supporting both open-vocabulary and closed-set industrial detection, providing valuable pre-training data for LVLMs in industrial scenarios. Furthermore, we propose a Refined Text-Visual Prompt Network (RTVPNet) that incorporates three key innovations: (1) an expert-assisted domain projection mechanism that enables rapid adaptation of general vision models to industrial domains, (2) an energy-based sparse sampling strategy that automatically generates refined visual prompts without manual intervention, and (3) a bidirectional text-visual interaction module that enhances cross-modal semantic alignment and understanding. Extensive experiments demonstrate that RTVPNet achieves state-of-the-art performance on MMIOC-1M, LVIS, and COCO benchmarks while maintaining computational efficiency. The dataset and code are available at https://github.com/hellozzk/MMIO.

Related papers

• Douyin Multimodal Embedding Model Technical Report [77.9018834072698]
  Multimodal representation learning is a cornerstone of modern AI.<n>We present Douyin Multimodal Embedding (DME), a model trained in two stages to combine both strengths.
  arXiv  Detail & Related papers  (2026-08-03T12:31:49Z)
• Zero-Shot Learning in Industrial Scenarios: New Large-Scale Benchmark, Challenges and Baseline [28.249460268707978]
  This paper proposes an open industrial dataset and a Refined Text-Visual Prompt (RTVP) for zero-shot industrial defect detection.<n> MMIO is the first large-scale multi-scenes pre-training dataset for industrial zero-shot learning.<n>RTVP automatically generates visual prompts directly from images and considers text-visual prompt interactions.
  arXiv  Detail & Related papers  (2026-06-06T03:48:12Z)
• FORGE: Fine-grained Multimodal Evaluation for Manufacturing Scenarios [58.34124792457706]
  The manufacturing sector is increasingly adopting Multimodal Large Language Models (MLLMs) to transition from simple perception to autonomous execution.<n>Progress is hindered by data scarcity and a lack of fine-grained domain semantics in existing datasets.<n>We first construct a high-quality dataset that combines realworld 2D images and 3D point clouds, annotated with fine-grained domain semantics.<n>We then evaluate 18 state-of-the-art MLLMs across three manufacturing tasks, namely verification, structural surface inspection, and assembly verification, revealing significant performance gaps.
  arXiv  Detail & Related papers  (2026-04-08T12:23:27Z)
• Referring Industrial Anomaly Segmentation [38.4461918800367]
  Industrial Anomaly Detection (IAD) is vital for manufacturing, yet traditional methods face challenges.<n>We propose Referring Industrial Anomaly (RIAS), a paradigm leveraging language to guide detection.<n>We introduce the MVTec-Ref dataset to support this, designed with diverse referring expressions and focusing on anomaly patterns.
  arXiv  Detail & Related papers  (2026-02-03T15:53:30Z)
• SSVP: Synergistic Semantic-Visual Prompting for Industrial Zero-Shot Anomaly Detection [55.54007781679915]
  We propose Synergistic Semantic-Visual Prompting (SSVP), that efficiently fuses diverse visual encodings to elevate model's fine-grained perception.<n>SSVP achieves state-of-the-art performance with 93.0% Image-AUROC and 92.2% Pixel-AUROC on MVTec-AD, significantly outperforming existing zero-shot approaches.
  arXiv  Detail & Related papers  (2026-01-14T04:42:19Z)
• Towards Open-Vocabulary Industrial Defect Understanding with a Large-Scale Multimodal Dataset [1.1470070927586018]
  We present IMDD-1M, the first large-scale Industrial Multimodal Defect dataset comprising 1,000,000 aligned image-text pairs.<n>This dataset enables a wide spectrum of applications, including classification, segmentation, retrieval, captioning, and generative modeling.
  arXiv  Detail & Related papers  (2025-12-30T11:45:22Z)
• OnePiece: Bringing Context Engineering and Reasoning to Industrial Cascade Ranking System [61.12400636463362]
  OnePiece is a unified framework that seamlessly integrates LLM-style context engineering and reasoning into both retrieval and ranking models.<n>OnePiece has been deployed in the main personalized search scenario of Shopee and achieves consistent online gains across different key business metrics.
  arXiv  Detail & Related papers  (2025-09-22T17:59:07Z)
• Leveraging Generative Models for Real-Time Query-Driven Text Summarization in Large-Scale Web Search [54.987957691350665]
  Query-Driven Text Summarization (QDTS) aims to generate concise and informative summaries from textual documents based on a given query.<n>Traditional extractive summarization models, based primarily on ranking candidate summary segments, have been the dominant approach in industrial applications.<n>We propose a novel framework to pioneer the application of generative models to address real-time QDTS in industrial web search.
  arXiv  Detail & Related papers  (2025-08-28T08:51:51Z)
• MVIP -- A Dataset and Methods for Application Oriented Multi-View and Multi-Modal Industrial Part Recognition [0.27309692684728604]
  MVIP is a novel dataset for multi-modal and multi-view application-oriented industrial part recognition.<n>Our main goal with MVIP is to study and push transferability of various state-of-the-art methods within related downstream tasks.
  arXiv  Detail & Related papers  (2025-02-21T13:22:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.