Industrial Language-Image Dataset (ILID): Adapting Vision Foundation Models for Industrial Settings

• URL: http://arxiv.org/abs/2406.09637v1
• Date: Fri, 14 Jun 2024 00:06:52 GMT
• Title: Industrial Language-Image Dataset (ILID): Adapting Vision Foundation Models for Industrial Settings
• Authors: Keno Moenck, Duc Trung Thieu, Julian Koch, Thorsten Schüppstuhl,
• Abstract summary: We introduce a pipeline to generate the Industrial Language-Image dataset (ILID) based on web-crawled data. We demonstrate effective self-supervised transfer learning and discussing downstream tasks after training on the cheaply acquired ILID.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In recent years, the upstream of Large Language Models (LLM) has also encouraged the computer vision community to work on substantial multimodal datasets and train models on a scale in a self-/semi-supervised manner, resulting in Vision Foundation Models (VFM), as, e.g., Contrastive Language-Image Pre-training (CLIP). The models generalize well and perform outstandingly on everyday objects or scenes, even on downstream tasks, tasks the model has not been trained on, while the application in specialized domains, as in an industrial context, is still an open research question. Here, fine-tuning the models or transfer learning on domain-specific data is unavoidable when objecting to adequate performance. In this work, we, on the one hand, introduce a pipeline to generate the Industrial Language-Image Dataset (ILID) based on web-crawled data; on the other hand, we demonstrate effective self-supervised transfer learning and discussing downstream tasks after training on the cheaply acquired ILID, which does not necessitate human labeling or intervention. With the proposed approach, we contribute by transferring approaches from state-of-the-art research around foundation models, transfer learning strategies, and applications to the industrial domain.

Related papers

• Evaluation and Comparison of Visual Language Models for Transportation Engineering Problems [16.49637074299509]
  We have explored state-of-the-art vision language models (VLM) for vision-based transportation engineering tasks. The image classification task involves congestion detection and crack identification, whereas, for object detection, helmet violations were identified. We have applied open-source models such as CLIP, BLIP, OWL-ViT, Llava-Next, and closed-source GPT-4o to evaluate the performance of these VLM models.
  arXiv  Detail & Related papers  (2024-09-03T20:24:37Z)
• Towards Synthetic Trace Generation of Modeling Operations using In-Context Learning Approach [1.8874331450711404]
  We propose a conceptual framework that combines modeling event logs, intelligent modeling assistants, and the generation of modeling operations. In particular, the architecture comprises modeling components that help the designer specify the system, record its operation within a graphical modeling environment, and automatically recommend relevant operations.
  arXiv  Detail & Related papers  (2024-08-26T13:26:44Z)
• AI Foundation Models in Remote Sensing: A Survey [6.036426846159163]
  This paper provides a comprehensive survey of foundation models in the remote sensing domain. We categorize these models based on their applications in computer vision and domain-specific tasks. We highlight emerging trends and the significant advancements achieved by these foundation models.
  arXiv  Detail & Related papers  (2024-08-06T22:39:34Z)
• Regularized Training with Generated Datasets for Name-Only Transfer of Vision-Language Models [36.59260354292177]
  Recent advancements in text-to-image generation have inspired researchers to generate datasets tailored for perception models using generative models. We aim to fine-tune vision-language models to a specific classification model without access to any real images. Despite the high fidelity of generated images, we observed a significant performance degradation when fine-tuning the model using the generated datasets.
  arXiv  Detail & Related papers  (2024-06-08T10:43:49Z)
• Exploring the Frontier of Vision-Language Models: A Survey of Current Methodologies and Future Directions [11.786387517781328]
  Vision-Language Models (VLMs) are advanced models that can tackle more intricate tasks such as image captioning and visual question answering. Our classification organizes VLMs into three distinct categories: models dedicated to vision-language understanding, models that process multimodal inputs to generate unimodal (textual) outputs and models that both accept and produce multimodal inputs and outputs. We meticulously dissect each model, offering an extensive analysis of its foundational architecture, training data sources, as well as its strengths and limitations wherever possible.
  arXiv  Detail & Related papers  (2024-02-20T18:57:34Z)
• Data-efficient Large Vision Models through Sequential Autoregression [58.26179273091461]
  We develop an efficient, autoregression-based vision model on a limited dataset. We demonstrate how this model achieves proficiency in a spectrum of visual tasks spanning both high-level and low-level semantic understanding. Our empirical evaluations underscore the model's agility in adapting to various tasks, heralding a significant reduction in the parameter footprint.
  arXiv  Detail & Related papers  (2024-02-07T13:41:53Z)
• UniDiff: Advancing Vision-Language Models with Generative and Discriminative Learning [86.91893533388628]
  This paper presents UniDiff, a unified multi-modal model that integrates image-text contrastive learning (ITC), text-conditioned image synthesis learning (IS), and reciprocal semantic consistency modeling (RSC) UniDiff demonstrates versatility in both multi-modal understanding and generative tasks.
  arXiv  Detail & Related papers  (2023-06-01T15:39:38Z)
• Pre-training Contextualized World Models with In-the-wild Videos for Reinforcement Learning [54.67880602409801]
  In this paper, we study the problem of pre-training world models with abundant in-the-wild videos for efficient learning of visual control tasks. We introduce Contextualized World Models (ContextWM) that explicitly separate context and dynamics modeling. Our experiments show that in-the-wild video pre-training equipped with ContextWM can significantly improve the sample efficiency of model-based reinforcement learning.
  arXiv  Detail & Related papers  (2023-05-29T14:29:12Z)
• Towards Efficient Task-Driven Model Reprogramming with Foundation Models [52.411508216448716]
  Vision foundation models exhibit impressive power, benefiting from the extremely large model capacity and broad training data. However, in practice, downstream scenarios may only support a small model due to the limited computational resources or efficiency considerations. This brings a critical challenge for the real-world application of foundation models: one has to transfer the knowledge of a foundation model to the downstream task.
  arXiv  Detail & Related papers  (2023-04-05T07:28:33Z)
• UViM: A Unified Modeling Approach for Vision with Learned Guiding Codes [91.24112204588353]
  We introduce UViM, a unified approach capable of modeling a wide range of computer vision tasks. In contrast to previous models, UViM has the same functional form for all tasks. We demonstrate the effectiveness of UViM on three diverse and challenging vision tasks.
  arXiv  Detail & Related papers  (2022-05-20T17:47:59Z)
• Model Reprogramming: Resource-Efficient Cross-Domain Machine Learning [65.268245109828]
  In data-rich domains such as vision, language, and speech, deep learning prevails to deliver high-performance task-specific models. Deep learning in resource-limited domains still faces multiple challenges including (i) limited data, (ii) constrained model development cost, and (iii) lack of adequate pre-trained models for effective finetuning. Model reprogramming enables resource-efficient cross-domain machine learning by repurposing a well-developed pre-trained model from a source domain to solve tasks in a target domain without model finetuning.
  arXiv  Detail & Related papers  (2022-02-22T02:33:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.