Towards General Industrial Intelligence: A Survey of Continual Large Models in Industrial IoT

• URL: http://arxiv.org/abs/2409.01207v2
• Date: Fri, 27 Dec 2024 01:23:05 GMT
• Title: Towards General Industrial Intelligence: A Survey of Continual Large Models in Industrial IoT
• Authors: Jiao Chen, Jiayi He, Fangfang Chen, Zuohong Lv, Jianhua Tang, Weihua Li, Zuozhu Liu, Howard H. Yang, Guangjie Han,
• Abstract summary: This survey explores the integration of IIoT with large models (LMs) and their potential applications in industrial environments. We focus on four primary types of industrial LMs: language-based, vision-based, time-series, and multimodal models. We analyze how IIoT provides abundant and diverse data resources, supporting the training and fine-tuning of LMs.
• Score: 25.16997700703974
• License:
• Abstract: Industrial AI is transitioning from traditional deep learning models to large-scale transformer-based architectures, with the Industrial Internet of Things (IIoT) playing a pivotal role. IIoT evolves from a simple data pipeline to an intelligent infrastructure, enabling and enhancing these advanced AI systems. This survey explores the integration of IIoT with large models (LMs) and their potential applications in industrial environments. We focus on four primary types of industrial LMs: language-based, vision-based, time-series, and multimodal models. The lifecycle of LMs is segmented into four critical phases: data foundation, model training, model connectivity, and continuous evolution. First, we analyze how IIoT provides abundant and diverse data resources, supporting the training and fine-tuning of LMs. Second, we discuss how IIoT offers an efficient training infrastructure in low-latency and bandwidth-optimized environments. Third, we highlight the deployment advantages of LMs within IIoT, emphasizing IIoT's role as a connectivity nexus fostering emergent intelligence through modular design, dynamic routing, and model merging to enhance system scalability and adaptability. Finally, we demonstrate how IIoT supports continual learning mechanisms, enabling LMs to adapt to dynamic industrial conditions and ensure long-term effectiveness. This paper underscores IIoT's critical role in the evolution of industrial intelligence with large models, offering a theoretical framework and actionable insights for future research.

Related papers

• LLMs Can Evolve Continually on Modality for X-Modal Reasoning [62.2874638875554]
  Existing methods rely heavily on modal-specific pretraining and joint-modal tuning, leading to significant computational burdens when expanding to new modalities. We propose PathWeave, a flexible and scalable framework with modal-Path sWitching and ExpAnsion abilities. PathWeave performs comparably to state-of-the-art MLLMs while concurrently reducing parameter training burdens by 98.73%.
  arXiv  Detail & Related papers  (2024-10-26T13:19:57Z)
• Long Term Memory: The Foundation of AI Self-Evolution [48.52678410533424]
  Large language models (LLMs) like GPTs, trained on vast datasets, have demonstrated impressive capabilities in language understanding, reasoning, and planning. Most studies focus on enhancing these models by training on ever-larger datasets to build more powerful foundation models. Unlike large-scale training, enabling models to evolve during inference is equally crucial, a process we refer to as AI self-evolution.
  arXiv  Detail & Related papers  (2024-10-21T06:09:30Z)
• Remaining Useful Life Prediction: A Study on Multidimensional Industrial Signal Processing and Efficient Transfer Learning Based on Large Language Models [6.118896920507198]
  This paper introduces an innovative regression framework utilizing large language models (LLMs) for RUL prediction. Experiments on the Turbofan engine's RUL prediction task show that the proposed model surpasses state-of-the-art (SOTA) methods. With minimal target domain data for fine-tuning, the model outperforms SOTA methods trained on full target domain data.
  arXiv  Detail & Related papers  (2024-10-04T04:21:53Z)
• NVLM: Open Frontier-Class Multimodal LLMs [64.00053046838225]
  We introduce NVLM 1.0, a family of frontier-class multimodal large language models (LLMs) that achieve state-of-the-art results on vision-language tasks. We propose a novel architecture that enhances both training efficiency and multimodal reasoning capabilities. We develop production-grade multimodality for the NVLM-1.0 models, enabling them to excel in vision-language tasks.
  arXiv  Detail & Related papers  (2024-09-17T17:59:06Z)
• Industrial Language-Image Dataset (ILID): Adapting Vision Foundation Models for Industrial Settings [0.0]
  We introduce a pipeline to generate the Industrial Language-Image dataset (ILID) based on web-crawled data. We demonstrate effective self-supervised transfer learning and discussing downstream tasks after training on the cheaply acquired ILID.
  arXiv  Detail & Related papers  (2024-06-14T00:06:52Z)
• Forging Vision Foundation Models for Autonomous Driving: Challenges, Methodologies, and Opportunities [59.02391344178202]
  Vision foundation models (VFMs) serve as potent building blocks for a wide range of AI applications. The scarcity of comprehensive training data, the need for multi-sensor integration, and the diverse task-specific architectures pose significant obstacles to the development of VFMs. This paper delves into the critical challenge of forging VFMs tailored specifically for autonomous driving, while also outlining future directions.
  arXiv  Detail & Related papers  (2024-01-16T01:57:24Z)
• When Parameter-efficient Tuning Meets General-purpose Vision-language Models [65.19127815275307]
  PETAL revolutionizes the training process by requiring only 0.5% of the total parameters, achieved through a unique mode approximation technique. Our experiments reveal that PETAL not only outperforms current state-of-the-art methods in most scenarios but also surpasses full fine-tuning models in effectiveness.
  arXiv  Detail & Related papers  (2023-12-16T17:13:08Z)
• A Multi-Stage Automated Online Network Data Stream Analytics Framework for IIoT Systems [10.350337750192997]
  We propose a novel Multi-Stage Automated Network Analytics (MSANA) framework for concept drift adaptation in IIoT systems. MSANA consists of dynamic data pre-processing, Drift-based Dynamic Feature Selection (DD-FS) method, dynamic model learning & selection, and Window-based Performance Weighted Probability Averaging Ensemble (W-PWPAE) model. It is a complete automated data stream analytics framework that enables automatic, effective, and efficient data analytics for IIoT systems in Industry 5.0.
  arXiv  Detail & Related papers  (2022-10-05T02:18:36Z)
• Federated Learning for Industrial Internet of Things in Future Industries [106.13524161081355]
  The Industrial Internet of Things (IIoT) offers promising opportunities to transform the operation of industrial systems. Recently, artificial intelligence (AI) has been widely utilized for realizing intelligent IIoT applications. Federated Learning (FL) is particularly attractive for intelligent IIoT networks by coordinating multiple IIoT devices and machines to perform AI training at the network edge.
  arXiv  Detail & Related papers  (2021-05-31T01:02:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.