A Cost-Sensitive Transformer Model for Prognostics Under Highly
Imbalanced Industrial Data
- URL: http://arxiv.org/abs/2402.08611v1
- Date: Tue, 16 Jan 2024 15:09:53 GMT
- Title: A Cost-Sensitive Transformer Model for Prognostics Under Highly
Imbalanced Industrial Data
- Authors: Ali Beikmohammadi, Mohammad Hosein Hamian, Neda Khoeyniha, Tony
Lindgren, Olof Steinert, and Sindri Magn\'usson
- Abstract summary: This paper introduces a novel cost-sensitive transformer model developed as part of a systematic workflow.
We observed a substantial enhancement in performance compared to state-of-the-art methods.
Our findings highlight the potential of our method in addressing the unique challenges of failure prediction in industrial settings.
- Score: 1.6492989697868894
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The rapid influx of data-driven models into the industrial sector has been
facilitated by the proliferation of sensor technology, enabling the collection
of vast quantities of data. However, leveraging these models for failure
detection and prognosis poses significant challenges, including issues like
missing values and class imbalances. Moreover, the cost sensitivity associated
with industrial operations further complicates the application of conventional
models in this context. This paper introduces a novel cost-sensitive
transformer model developed as part of a systematic workflow, which also
integrates a hybrid resampler and a regression-based imputer. After subjecting
our approach to rigorous testing using the APS failure dataset from Scania
trucks and the SECOM dataset, we observed a substantial enhancement in
performance compared to state-of-the-art methods. Moreover, we conduct an
ablation study to analyze the contributions of different components in our
proposed method. Our findings highlight the potential of our method in
addressing the unique challenges of failure prediction in industrial settings,
thereby contributing to enhanced reliability and efficiency in industrial
operations.
Related papers
- Towards Resource-Efficient Federated Learning in Industrial IoT for Multivariate Time Series Analysis [50.18156030818883]
Anomaly and missing data constitute a thorny problem in industrial applications.
Deep learning enabled anomaly detection has emerged as a critical direction.
The data collected in edge devices contain user privacy.
arXiv Detail & Related papers (2024-11-06T15:38:31Z) - AAD-LLM: Adaptive Anomaly Detection Using Large Language Models [35.286105732902065]
The research aims to improve the transferability of anomaly detection models by leveraging Large Language Models (LLMs)
The research also seeks to enable more collaborative decision-making between the model and plant operators.
arXiv Detail & Related papers (2024-11-01T13:43:28Z) - Sustainable Diffusion-based Incentive Mechanism for Generative AI-driven Digital Twins in Industrial Cyber-Physical Systems [65.22300383287904]
Industrial Cyber-Physical Systems (ICPSs) are an integral component of modern manufacturing and industries.
By digitizing data throughout the product life cycle, Digital Twins (DTs) in ICPSs enable a shift from current industrial infrastructures to intelligent and adaptive infrastructures.
mechanisms that leverage sensing Industrial Internet of Things (IIoT) devices to share data for the construction of DTs are susceptible to adverse selection problems.
arXiv Detail & Related papers (2024-08-02T10:47:10Z) - A Predictive Model Based on Transformer with Statistical Feature Embedding in Manufacturing Sensor Dataset [2.07180164747172]
This study proposes a novel predictive model based on the Transformer, utilizing statistical feature embedding and window positional encoding.
The model's performance is evaluated in two problems: fault detection and virtual metrology, showing superior results compared to baseline models.
The results support the model's applicability across various manufacturing industries, demonstrating its potential for enhancing process management and yield.
arXiv Detail & Related papers (2024-07-09T08:59:27Z) - Unveiling the Flaws: Exploring Imperfections in Synthetic Data and Mitigation Strategies for Large Language Models [89.88010750772413]
Synthetic data has been proposed as a solution to address the issue of high-quality data scarcity in the training of large language models (LLMs)
Our work delves into these specific flaws associated with question-answer (Q-A) pairs, a prevalent type of synthetic data, and presents a method based on unlearning techniques to mitigate these flaws.
Our work has yielded key insights into the effective use of synthetic data, aiming to promote more robust and efficient LLM training.
arXiv Detail & Related papers (2024-06-18T08:38:59Z) - Comparative Analysis of Transformers for Modeling Tabular Data: A
Casestudy using Industry Scale Dataset [1.0758036046280266]
The study conducts an extensive examination of various transformer-based models using both synthetic datasets and the default prediction Kaggle dataset (2022) from American Express.
The paper presents crucial insights into optimal data pre-processing, compares pre-training and direct supervised learning methods, discusses strategies for managing categorical and numerical features, and highlights trade-offs between computational resources and performance.
arXiv Detail & Related papers (2023-11-24T08:16:39Z) - Predictive Maintenance Model Based on Anomaly Detection in Induction
Motors: A Machine Learning Approach Using Real-Time IoT Data [0.0]
In this work, we demonstrate a novel anomaly detection system on induction motors used in pumps, compressors, fans, and other industrial machines.
We use a combination of pre-processing techniques and machine learning (ML) models with a low computational cost.
arXiv Detail & Related papers (2023-10-15T18:43:45Z) - End-to-End Meta-Bayesian Optimisation with Transformer Neural Processes [52.818579746354665]
This paper proposes the first end-to-end differentiable meta-BO framework that generalises neural processes to learn acquisition functions via transformer architectures.
We enable this end-to-end framework with reinforcement learning (RL) to tackle the lack of labelled acquisition data.
arXiv Detail & Related papers (2023-05-25T10:58:46Z) - Soft Sensing Regression Model: from Sensor to Wafer Metrology
Forecasting [2.8992789044888436]
This work focuses on the task of soft sensing regression, which uses sensor data to predict impending inspection measurements.
We proposed an LSTM-based regressor and designed two loss functions for model training.
The experimental results demonstrated that the proposed model can achieve accurate and early prediction of various types of inspections in complicated manufacturing processes.
arXiv Detail & Related papers (2023-01-21T16:54:05Z) - Deep Learning based pipeline for anomaly detection and quality
enhancement in industrial binder jetting processes [68.8204255655161]
Anomaly detection describes methods of finding abnormal states, instances or data points that differ from a normal value space.
This paper contributes to a data-centric way of approaching artificial intelligence in industrial production.
arXiv Detail & Related papers (2022-09-21T08:14:34Z) - On a Uniform Causality Model for Industrial Automation [61.303828551910634]
A Uniform Causality Model for various application areas of industrial automation is proposed.
The resulting model describes the behavior of Cyber-Physical Systems mathematically.
It is shown that the model can work as a basis for the application of new approaches in industrial automation that focus on machine learning.
arXiv Detail & Related papers (2022-09-20T11:23:51Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.