Generative Pre-Training of Time-Series Data for Unsupervised Fault Detection in Semiconductor Manufacturing

• URL: http://arxiv.org/abs/2309.11427v2
• Date: Wed, 27 Mar 2024 14:02:57 GMT
• Title: Generative Pre-Training of Time-Series Data for Unsupervised Fault Detection in Semiconductor Manufacturing
• Authors: Sewoong Lee, JinKyou Choi, Min Su Kim,
• Abstract summary: This paper introduces TRACE-GPT, which stands for Time-seRies Anomaly-detection with Convolutional Embedding and Generative Pre-trained Transformers. Our model has the highest F1 score at Equal Error Rate (EER) across all datasets and is only 0.026 below the supervised state-of-the-art baseline on the open dataset.
• Score: 20.71342655519934
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper introduces TRACE-GPT, which stands for Time-seRies Anomaly-detection with Convolutional Embedding and Generative Pre-trained Transformers. TRACE-GPT is designed to pre-train univariate time-series sensor data and detect faults on unlabeled datasets in semiconductor manufacturing. In semiconductor industry, classifying abnormal time-series sensor data from normal data is important because it is directly related to wafer defect. However, small, unlabeled, and even mixed training data without enough anomalies make classification tasks difficult. In this research, we capture features of time-series data with temporal convolutional embedding and Generative Pre-trained Transformer (GPT) to classify abnormal sequences from normal sequences using cross entropy loss. We prove that our model shows better performance than previous unsupervised models with both an open dataset, the University of California Riverside (UCR) time-series classification archive, and the process log of our Chemical Vapor Deposition (CVD) equipment. Our model has the highest F1 score at Equal Error Rate (EER) across all datasets and is only 0.026 below the supervised state-of-the-art baseline on the open dataset.

Related papers

• Leveraging Latent Diffusion Models for Training-Free In-Distribution Data Augmentation for Surface Defect Detection [9.784793380119806]
  We introduce DIAG, a training-free Diffusion-based In-distribution Anomaly Generation pipeline for data augmentation. Unlike conventional image generation techniques, we implement a human-in-the-loop pipeline, where domain experts provide multimodal guidance to the model. We demonstrate the efficacy and versatility of DIAG with respect to state-of-the-art data augmentation approaches on the challenging KSDD2 dataset.
  arXiv  Detail & Related papers  (2024-07-04T14:28:52Z)
• EntropyStop: Unsupervised Deep Outlier Detection with Loss Entropy [19.154826741973277]
  We propose a zero-label entropy metric named Loss Entropy for loss distribution, enabling us to infer optimal stopping points for training without labels. We also develop an automated early-stopping algorithm, EntropyStop, which halts training when loss entropy suggests the maximum model detection capability.
  arXiv  Detail & Related papers  (2024-05-21T05:17:43Z)
• Graph Spatiotemporal Process for Multivariate Time Series Anomaly Detection with Missing Values [67.76168547245237]
  We introduce a novel framework called GST-Pro, which utilizes a graphtemporal process and anomaly scorer to detect anomalies. Our experimental results show that the GST-Pro method can effectively detect anomalies in time series data and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-01-11T10:10:16Z)
• LARA: A Light and Anti-overfitting Retraining Approach for Unsupervised Time Series Anomaly Detection [49.52429991848581]
  We propose a Light and Anti-overfitting Retraining Approach (LARA) for deep variational auto-encoder based time series anomaly detection methods (VAEs) This work aims to make three novel contributions: 1) the retraining process is formulated as a convex problem and can converge at a fast rate as well as prevent overfitting; 2) designing a ruminate block, which leverages the historical data without the need to store them; and 3) mathematically proving that when fine-tuning the latent vector and reconstructed data, the linear formations can achieve the least adjusting errors between the ground truths and the fine-tuned ones.
  arXiv  Detail & Related papers  (2023-10-09T12:36:16Z)
• Imbalanced Aircraft Data Anomaly Detection [103.01418862972564]
  Anomaly detection in temporal data from sensors under aviation scenarios is a practical but challenging task. We propose a Graphical Temporal Data Analysis framework. It consists three modules, named Series-to-Image (S2I), Cluster-based Resampling Approach using Euclidean Distance (CRD) and Variance-Based Loss (VBL)
  arXiv  Detail & Related papers  (2023-05-17T09:37:07Z)
• AnomalyBERT: Self-Supervised Transformer for Time Series Anomaly Detection using Data Degradation Scheme [0.7216399430290167]
  Anomaly detection task for time series, especially for unlabeled data, has been a challenging problem. We address it by applying a suitable data degradation scheme to self-supervised model training. Inspired by the self-attention mechanism, we design a Transformer-based architecture to recognize the temporal context.
  arXiv  Detail & Related papers  (2023-05-08T05:42:24Z)
• Are we certain it's anomalous? [57.729669157989235]
  Anomaly detection in time series is a complex task since anomalies are rare due to highly non-linear temporal correlations. Here we propose the novel use of Hyperbolic uncertainty for Anomaly Detection (HypAD) HypAD learns self-supervisedly to reconstruct the input signal.
  arXiv  Detail & Related papers  (2022-11-16T21:31:39Z)
• DEGAN: Time Series Anomaly Detection using Generative Adversarial Network Discriminators and Density Estimation [0.0]
  We have proposed an unsupervised Generative Adversarial Network (GAN)-based anomaly detection framework, DEGAN. It relies solely on normal time series data as input to train a well-configured discriminator (D) into a standalone anomaly predictor.
  arXiv  Detail & Related papers  (2022-10-05T04:32:12Z)
• ALGAN: Anomaly Detection by Generating Pseudo Anomalous Data via Latent Variables [17.53032543377636]
  We propose an Anomalous Latent variable Generative Adversarial Network (ALGAN) in which the GAN generator produces pseudo-anomalous data as well as fake-normal data. The proposed ALGAN exhibited an AUROC comparable to state-of-the-art methods while achieving a much faster prediction time.
  arXiv  Detail & Related papers  (2022-02-21T14:53:05Z)
• Deep Transformer Networks for Time Series Classification: The NPP Safety Case [59.20947681019466]
  An advanced temporal neural network referred to as the Transformer is used within a supervised learning fashion to model the time-dependent NPP simulation data. The Transformer can learn the characteristics of the sequential data and yield promising performance with approximately 99% classification accuracy on the testing dataset.
  arXiv  Detail & Related papers  (2021-04-09T14:26:25Z)
• TadGAN: Time Series Anomaly Detection Using Generative Adversarial Networks [73.01104041298031]
  TadGAN is an unsupervised anomaly detection approach built on Generative Adversarial Networks (GANs) To capture the temporal correlations of time series, we use LSTM Recurrent Neural Networks as base models for Generators and Critics. To demonstrate the performance and generalizability of our approach, we test several anomaly scoring techniques and report the best-suited one.
  arXiv  Detail & Related papers  (2020-09-16T15:52:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.