A Cost-Sensitive Transformer Model for Prognostics Under Highly Imbalanced Industrial Data

• URL: http://arxiv.org/abs/2402.08611v1
• Date: Tue, 16 Jan 2024 15:09:53 GMT
• Title: A Cost-Sensitive Transformer Model for Prognostics Under Highly Imbalanced Industrial Data
• Authors: Ali Beikmohammadi, Mohammad Hosein Hamian, Neda Khoeyniha, Tony Lindgren, Olof Steinert, and Sindri Magn\'usson
• Abstract summary: This paper introduces a novel cost-sensitive transformer model developed as part of a systematic workflow. We observed a substantial enhancement in performance compared to state-of-the-art methods. Our findings highlight the potential of our method in addressing the unique challenges of failure prediction in industrial settings.
• Score: 1.6492989697868894
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The rapid influx of data-driven models into the industrial sector has been facilitated by the proliferation of sensor technology, enabling the collection of vast quantities of data. However, leveraging these models for failure detection and prognosis poses significant challenges, including issues like missing values and class imbalances. Moreover, the cost sensitivity associated with industrial operations further complicates the application of conventional models in this context. This paper introduces a novel cost-sensitive transformer model developed as part of a systematic workflow, which also integrates a hybrid resampler and a regression-based imputer. After subjecting our approach to rigorous testing using the APS failure dataset from Scania trucks and the SECOM dataset, we observed a substantial enhancement in performance compared to state-of-the-art methods. Moreover, we conduct an ablation study to analyze the contributions of different components in our proposed method. Our findings highlight the potential of our method in addressing the unique challenges of failure prediction in industrial settings, thereby contributing to enhanced reliability and efficiency in industrial operations.

Related papers

• Towards Resource-Efficient Federated Learning in Industrial IoT for Multivariate Time Series Analysis [50.18156030818883]
  Anomaly and missing data constitute a thorny problem in industrial applications. Deep learning enabled anomaly detection has emerged as a critical direction. The data collected in edge devices contain user privacy.
  arXiv  Detail & Related papers  (2024-11-06T15:38:31Z)
• AAD-LLM: Adaptive Anomaly Detection Using Large Language Models [35.286105732902065]
  The research aims to improve the transferability of anomaly detection models by leveraging Large Language Models (LLMs) The research also seeks to enable more collaborative decision-making between the model and plant operators.
  arXiv  Detail & Related papers  (2024-11-01T13:43:28Z)
• Sustainable Diffusion-based Incentive Mechanism for Generative AI-driven Digital Twins in Industrial Cyber-Physical Systems [65.22300383287904]
  Industrial Cyber-Physical Systems (ICPSs) are an integral component of modern manufacturing and industries. By digitizing data throughout the product life cycle, Digital Twins (DTs) in ICPSs enable a shift from current industrial infrastructures to intelligent and adaptive infrastructures. mechanisms that leverage sensing Industrial Internet of Things (IIoT) devices to share data for the construction of DTs are susceptible to adverse selection problems.
  arXiv  Detail & Related papers  (2024-08-02T10:47:10Z)
• A Predictive Model Based on Transformer with Statistical Feature Embedding in Manufacturing Sensor Dataset [2.07180164747172]
  This study proposes a novel predictive model based on the Transformer, utilizing statistical feature embedding and window positional encoding. The model's performance is evaluated in two problems: fault detection and virtual metrology, showing superior results compared to baseline models. The results support the model's applicability across various manufacturing industries, demonstrating its potential for enhancing process management and yield.
  arXiv  Detail & Related papers  (2024-07-09T08:59:27Z)
• Unveiling the Flaws: Exploring Imperfections in Synthetic Data and Mitigation Strategies for Large Language Models [89.88010750772413]
  Synthetic data has been proposed as a solution to address the issue of high-quality data scarcity in the training of large language models (LLMs) Our work delves into these specific flaws associated with question-answer (Q-A) pairs, a prevalent type of synthetic data, and presents a method based on unlearning techniques to mitigate these flaws. Our work has yielded key insights into the effective use of synthetic data, aiming to promote more robust and efficient LLM training.
  arXiv  Detail & Related papers  (2024-06-18T08:38:59Z)
• Comparative Analysis of Transformers for Modeling Tabular Data: A Casestudy using Industry Scale Dataset [1.0758036046280266]
  The study conducts an extensive examination of various transformer-based models using both synthetic datasets and the default prediction Kaggle dataset (2022) from American Express. The paper presents crucial insights into optimal data pre-processing, compares pre-training and direct supervised learning methods, discusses strategies for managing categorical and numerical features, and highlights trade-offs between computational resources and performance.
  arXiv  Detail & Related papers  (2023-11-24T08:16:39Z)
• Predictive Maintenance Model Based on Anomaly Detection in Induction Motors: A Machine Learning Approach Using Real-Time IoT Data [0.0]
  In this work, we demonstrate a novel anomaly detection system on induction motors used in pumps, compressors, fans, and other industrial machines. We use a combination of pre-processing techniques and machine learning (ML) models with a low computational cost.
  arXiv  Detail & Related papers  (2023-10-15T18:43:45Z)
• End-to-End Meta-Bayesian Optimisation with Transformer Neural Processes [52.818579746354665]
  This paper proposes the first end-to-end differentiable meta-BO framework that generalises neural processes to learn acquisition functions via transformer architectures. We enable this end-to-end framework with reinforcement learning (RL) to tackle the lack of labelled acquisition data.
  arXiv  Detail & Related papers  (2023-05-25T10:58:46Z)
• Soft Sensing Regression Model: from Sensor to Wafer Metrology Forecasting [2.8992789044888436]
  This work focuses on the task of soft sensing regression, which uses sensor data to predict impending inspection measurements. We proposed an LSTM-based regressor and designed two loss functions for model training. The experimental results demonstrated that the proposed model can achieve accurate and early prediction of various types of inspections in complicated manufacturing processes.
  arXiv  Detail & Related papers  (2023-01-21T16:54:05Z)
• Deep Learning based pipeline for anomaly detection and quality enhancement in industrial binder jetting processes [68.8204255655161]
  Anomaly detection describes methods of finding abnormal states, instances or data points that differ from a normal value space. This paper contributes to a data-centric way of approaching artificial intelligence in industrial production.
  arXiv  Detail & Related papers  (2022-09-21T08:14:34Z)
• On a Uniform Causality Model for Industrial Automation [61.303828551910634]
  A Uniform Causality Model for various application areas of industrial automation is proposed. The resulting model describes the behavior of Cyber-Physical Systems mathematically. It is shown that the model can work as a basis for the application of new approaches in industrial automation that focus on machine learning.
  arXiv  Detail & Related papers  (2022-09-20T11:23:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.