Zero-Shot Learning in Industrial Scenarios: New Large-Scale Benchmark, Challenges and Baseline

• URL: http://arxiv.org/abs/2606.07965v1
• Date: Sat, 06 Jun 2026 03:48:12 GMT
• Title: Zero-Shot Learning in Industrial Scenarios: New Large-Scale Benchmark, Challenges and Baseline
• Abstract summary: This paper proposes an open industrial dataset and a Refined Text-Visual Prompt (RTVP) for zero-shot industrial defect detection.<n> MMIO is the first large-scale multi-scenes pre-training dataset for industrial zero-shot learning.<n>RTVP automatically generates visual prompts directly from images and considers text-visual prompt interactions.
• Score: 28.249460268707978
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Visual Language Models (LVLMs) have achieved remarkable success in vision tasks. However, the significant differences between industrial and natural scenes make applying LVLMs challenging. Existing LVLMs rely on user-provided prompts to segment objects. This often leads to suboptimal performance due to the inclusion of irrelevant pixels. In addition, the scarcity of data also makes the application of LVLMs in industrial scenarios remain unexplored. To fill this gap, this paper proposes an open industrial dataset and a Refined Text-Visual Prompt (RTVP) for zero-shot industrial defect detection. First, this paper constructs the Multi-Modal Industrial Open Dataset (MMIO) containing 80K+ samples. MMIO contains diverse industrial categories, including 6 super categories and 18 subcategories. MMIO is the first large-scale multi-scenes pre-training dataset for industrial zero-shot learning, and provides valuable training data for open models in future industrial scenarios. Based on MMIO, this paper provides a RTVP specifically for industrial zero-shot tasks. RTVP has two significant advantages: First, this paper designs an expert-guided large model domain adaptation mechanism and designs an industrial zero-shot method based on Mobile-SAM, which enhances the generalization ability of large models in industrial scenarios. Second, RTVP automatically generates visual prompts directly from images and considers text-visual prompt interactions ignored by previous LVLM, improving visual and textual content understanding. RTVP achieves SOTA with 42.2% and 24.7% AP in zero-shot and closed scenes of MMIO.

Related papers

• Unification of Closed-Open Industrial Detection Scenarios: New Large-Scale Benchmarks,Challenges and Baselines [50.471211036005286]
  We introduce a Large-Scale Multi-Modal Industrial Open-Closed benchmark (MMIOC-1M)<n> MMIOC-1M is the first unified largest benchmark supporting both open-vocabulary and closed-set industrial detection.<n>We propose a Refined Text-Visual Prompt Network (RTVPNet) that incorporates three key innovations.
  arXiv  Detail & Related papers  (2026-06-06T03:06:10Z)
• OmniVTG: A Large-Scale Dataset and Training Paradigm for Open-World Video Temporal Grounding [55.29748680163419]
  Video Temporal Grounding (VTG) struggles in open-world settings due to limited dataset scale and semantic diversity.<n>We introduce OmniVTG, a new large-scale dataset for open-world VTG.<n>We train the MLLM to first predict, then use its understanding capabilities to reflect on and refine its own predictions.
  arXiv  Detail & Related papers  (2026-04-28T06:34:19Z)
• Open-vocabulary 3D scene perception in industrial environments [0.0]
  Recent open-vocabulary methods, leveraging 2D Vision-Language Foundation Models (VLFMs), target this task.<n>We first demonstrate that such models fail to generalize, performing poorly on common industrial objects.<n>We propose a training-free, open-vocabulary 3D perception pipeline that overcomes this limitation.
  arXiv  Detail & Related papers  (2026-02-23T13:22:51Z)
• ZERO: Industry-ready Vision Foundation Model with Multi-modal Prompts [1.2744523252873352]
  Superb AI introduces ZERO, an industry-ready vision foundation model.<n> ZERO is trained on a compact yet representative 0.9 million annotated samples from a proprietary billion-scale industrial dataset.<n> ZERO is the first vision foundation model explicitly built for domain-specific, zero-shot industrial applications.
  arXiv  Detail & Related papers  (2025-07-06T07:03:27Z)
• Benchmarking Large Vision-Language Models on Fine-Grained Image Tasks: A Comprehensive Evaluation [53.84282335629258]
  We introduce a comprehensive fine-grained evaluation benchmark, i.e., FG-BMK, comprising 1.01 million questions and 0.33 million images.<n>Our evaluation systematically examines LVLMs from both human-oriented and machine-oriented perspectives.<n>We uncover key findings regarding the influence of training paradigms, modality alignment, perturbation susceptibility, and fine-grained category reasoning on task performance.
  arXiv  Detail & Related papers  (2025-04-21T09:30:41Z)
• MVIP -- A Dataset and Methods for Application Oriented Multi-View and Multi-Modal Industrial Part Recognition [0.27309692684728604]
  MVIP is a novel dataset for multi-modal and multi-view application-oriented industrial part recognition.<n>Our main goal with MVIP is to study and push transferability of various state-of-the-art methods within related downstream tasks.
  arXiv  Detail & Related papers  (2025-02-21T13:22:29Z)
• MMAD: A Comprehensive Benchmark for Multimodal Large Language Models in Industrial Anomaly Detection [66.05200339481115]
  We present MMAD, the first-ever full-spectrum MLLMs benchmark in industrial anomaly detection.<n>We defined seven key subtasks of MLLMs in industrial inspection and designed a novel pipeline to generate the MMAD dataset.<n>With MMAD, we have conducted a comprehensive, quantitative evaluation of various state-of-the-art MLLMs.
  arXiv  Detail & Related papers  (2024-10-12T09:16:09Z)
• IPAD: Industrial Process Anomaly Detection Dataset [71.39058003212614]
  Video anomaly detection (VAD) is a challenging task aiming to recognize anomalies in video frames. We propose a new dataset, IPAD, specifically designed for VAD in industrial scenarios. This dataset covers 16 different industrial devices and contains over 6 hours of both synthetic and real-world video footage.
  arXiv  Detail & Related papers  (2024-04-23T13:38:01Z)
• LVLM-eHub: A Comprehensive Evaluation Benchmark for Large Vision-Language Models [55.304181390027274]
  This paper presents a comprehensive evaluation of publicly available large multimodal models by building a LVLM evaluation Hub (LVLM-eHub) Our LVLM-eHub consists of $8$ representative LVLMs such as InstructBLIP and MiniGPT-4, which are thoroughly evaluated by a quantitative capability evaluation and an online arena platform. The study reveals several innovative findings. First, instruction-tuned LVLM with massive in-domain data such as InstructBLIP heavily overfits many existing tasks, generalizing poorly in the open-world scenario.
  arXiv  Detail & Related papers  (2023-06-15T16:39:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.