A Comprehensive Survey of Deep Transfer Learning for Anomaly Detection
in Industrial Time Series: Methods, Applications, and Directions
- URL: http://arxiv.org/abs/2307.05638v2
- Date: Wed, 10 Jan 2024 22:15:06 GMT
- Title: A Comprehensive Survey of Deep Transfer Learning for Anomaly Detection
in Industrial Time Series: Methods, Applications, and Directions
- Authors: Peng Yan, Ahmed Abdulkadir, Paul-Philipp Luley, Matthias Rosenthal,
Gerrit A. Schatte, Benjamin F. Grewe, Thilo Stadelmann
- Abstract summary: The monitoring of industrial processes has the potential to enhance efficiency and optimize quality.
Deep learning, with its capacity to discern non-trivial patterns within large datasets, plays a pivotal role in this process.
It is impractical to acquire large-scale labeled data for standard deep learning training for every slightly different case anew.
Deep transfer learning offers a solution to this problem.
- Score: 5.759456719890725
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Automating the monitoring of industrial processes has the potential to
enhance efficiency and optimize quality by promptly detecting abnormal events
and thus facilitating timely interventions. Deep learning, with its capacity to
discern non-trivial patterns within large datasets, plays a pivotal role in
this process. Standard deep learning methods are suitable to solve a specific
task given a specific type of data. During training, deep learning demands
large volumes of labeled data. However, due to the dynamic nature of the
industrial processes and environment, it is impractical to acquire large-scale
labeled data for standard deep learning training for every slightly different
case anew. Deep transfer learning offers a solution to this problem. By
leveraging knowledge from related tasks and accounting for variations in data
distributions, the transfer learning framework solves new tasks with little or
even no additional labeled data. The approach bypasses the need to retrain a
model from scratch for every new setup and dramatically reduces the labeled
data requirement. This survey first provides an in-depth review of deep
transfer learning, examining the problem settings of transfer learning and
classifying the prevailing deep transfer learning methods. Moreover, we delve
into applications of deep transfer learning in the context of a broad spectrum
of time series anomaly detection tasks prevalent in primary industrial domains,
e.g., manufacturing process monitoring, predictive maintenance, energy
management, and infrastructure facility monitoring. We discuss the challenges
and limitations of deep transfer learning in industrial contexts and conclude
the survey with practical directions and actionable suggestions to address the
need to leverage diverse time series data for anomaly detection in an
increasingly dynamic production environment.
Related papers
- Mind the Interference: Retaining Pre-trained Knowledge in Parameter Efficient Continual Learning of Vision-Language Models [79.28821338925947]
Domain-Class Incremental Learning is a realistic but challenging continual learning scenario.
To handle these diverse tasks, pre-trained Vision-Language Models (VLMs) are introduced for their strong generalizability.
This incurs a new problem: the knowledge encoded in the pre-trained VLMs may be disturbed when adapting to new tasks, compromising their inherent zero-shot ability.
Existing methods tackle it by tuning VLMs with knowledge distillation on extra datasets, which demands heavy overhead.
We propose the Distribution-aware Interference-free Knowledge Integration (DIKI) framework, retaining pre-trained knowledge of
arXiv Detail & Related papers (2024-07-07T12:19:37Z) - Deep Learning for Trajectory Data Management and Mining: A Survey and Beyond [58.63558696061679]
Trajectory computing is crucial in various practical applications such as location services, urban traffic, and public safety.
We present a review of development and recent advances in deep learning for trajectory computing (DL4Traj)
Notably, we encapsulate recent advancements in Large Language Models (LLMs) that hold potential to augment trajectory computing.
arXiv Detail & Related papers (2024-03-21T05:57:27Z) - Robust Machine Learning by Transforming and Augmenting Imperfect
Training Data [6.928276018602774]
This thesis explores several data sensitivities of modern machine learning.
We first discuss how to prevent ML from codifying prior human discrimination measured in the training data.
We then discuss the problem of learning from data containing spurious features, which provide predictive fidelity during training but are unreliable upon deployment.
arXiv Detail & Related papers (2023-12-19T20:49:28Z) - Deep Learning based pipeline for anomaly detection and quality
enhancement in industrial binder jetting processes [68.8204255655161]
Anomaly detection describes methods of finding abnormal states, instances or data points that differ from a normal value space.
This paper contributes to a data-centric way of approaching artificial intelligence in industrial production.
arXiv Detail & Related papers (2022-09-21T08:14:34Z) - Consecutive Pretraining: A Knowledge Transfer Learning Strategy with
Relevant Unlabeled Data for Remote Sensing Domain [25.84756140221655]
ConSecutive PreTraining (CSPT) is proposed based on the idea of not stopping pretraining in natural language processing (NLP)
The proposed CSPT also can release the huge potential of unlabeled data for task-aware model training.
The results show that by utilizing the proposed CSPT for task-aware model training, almost all downstream tasks in RSD can outperform the previous method of supervised pretraining-then-fine-tuning.
arXiv Detail & Related papers (2022-07-08T12:32:09Z) - Understanding the World Through Action [91.3755431537592]
I will argue that a general, principled, and powerful framework for utilizing unlabeled data can be derived from reinforcement learning.
I will discuss how such a procedure is more closely aligned with potential downstream tasks.
arXiv Detail & Related papers (2021-10-24T22:33:52Z) - Towards Deep Industrial Transfer Learning for Anomaly Detection on Time
Series Data [0.0]
Deep learning promises performant anomaly detection on time-variant datasets.
Deep transfer learning offers mitigation by letting algorithms built upon previous knowledge from different tasks or locations.
arXiv Detail & Related papers (2021-06-09T08:58:56Z) - Data-efficient Weakly-supervised Learning for On-line Object Detection
under Domain Shift in Robotics [24.878465999976594]
Several object detection methods have been proposed in the literature, the vast majority based on Deep Convolutional Neural Networks (DCNNs)
These methods have important limitations for robotics: Learning solely on off-line data may introduce biases, and prevents adaptation to novel tasks.
In this work, we investigate how weakly-supervised learning can cope with these problems.
arXiv Detail & Related papers (2020-12-28T16:36:11Z) - Parrot: Data-Driven Behavioral Priors for Reinforcement Learning [79.32403825036792]
We propose a method for pre-training behavioral priors that can capture complex input-output relationships observed in successful trials.
We show how this learned prior can be used for rapidly learning new tasks without impeding the RL agent's ability to try out novel behaviors.
arXiv Detail & Related papers (2020-11-19T18:47:40Z) - A Survey on Self-supervised Pre-training for Sequential Transfer
Learning in Neural Networks [1.1802674324027231]
Self-supervised pre-training for transfer learning is becoming an increasingly popular technique to improve state-of-the-art results using unlabeled data.
We provide an overview of the taxonomy for self-supervised learning and transfer learning, and highlight some prominent methods for designing pre-training tasks across different domains.
arXiv Detail & Related papers (2020-07-01T22:55:48Z) - A System for Real-Time Interactive Analysis of Deep Learning Training [66.06880335222529]
Currently available systems are limited to monitoring only the logged data that must be specified before the training process starts.
We present a new system that enables users to perform interactive queries on live processes generating real-time information.
arXiv Detail & Related papers (2020-01-05T11:33:31Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.