RmGPT: A Foundation Model with Generative Pre-trained Transformer for Fault Diagnosis and Prognosis in Rotating Machinery

• URL: http://arxiv.org/abs/2409.17604v2
• Date: Thu, 12 Jun 2025 13:09:56 GMT
• Title: RmGPT: A Foundation Model with Generative Pre-trained Transformer for Fault Diagnosis and Prognosis in Rotating Machinery
• Authors: Yilin Wang, Yifei Yu, Kong Sun, Peixuan Lei, Yuxuan Zhang, Enrico Zio, Aiguo Xia, Yuanxiang Li,
• Abstract summary: Current methods of Prognostics and Health Management (PHM) often rely on task-specific models.<n>Inspired by advancements in generative pretrained models, we propose RmGPT, a unified model for diagnosis and prognosis tasks.<n>RmGPT significantly outperforms state-of-the-art algorithms, achieving near-perfect accuracy in diagnosis tasks and exceptionally low errors in prognosis tasks.
• Score: 20.52039868199533
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In industry, the reliability of rotating machinery is critical for production efficiency and safety. Current methods of Prognostics and Health Management (PHM) often rely on task-specific models, which face significant challenges in handling diverse datasets with varying signal characteristics, fault modes and operating conditions. Inspired by advancements in generative pretrained models, we propose RmGPT, a unified model for diagnosis and prognosis tasks. RmGPT introduces a novel generative token-based framework, incorporating Signal Tokens, Prompt Tokens, Time-Frequency Task Tokens and Fault Tokens to handle heterogeneous data within a unified model architecture. We leverage self-supervised learning for robust feature extraction and introduce a next signal token prediction pretraining strategy, alongside efficient prompt learning for task-specific adaptation. Extensive experiments demonstrate that RmGPT significantly outperforms state-of-the-art algorithms, achieving near-perfect accuracy in diagnosis tasks and exceptionally low errors in prognosis tasks. Notably, RmGPT excels in few-shot learning scenarios, achieving 82\% accuracy in 16-class one-shot experiments, highlighting its adaptability and robustness. This work establishes RmGPT as a powerful PHM foundation model for rotating machinery, advancing the scalability and generalizability of PHM solutions. \textbf{Code is available at: https://github.com/Pandalin98/RmGPT.

Related papers

• Pretraining Transformer-Based Models on Diffusion-Generated Synthetic Graphs for Alzheimer's Disease Prediction [0.0]
  We propose a Transformer-based diagnostic framework that combines synthetic data generation with graph representation learning and transfer learning.<n>A class-conditional denoising diffusion probabilistic model (DDPM) is trained on the real-world NACC dataset to generate a large synthetic cohort.<n> Modality-specific Graph Transformer encoders are first pretrained on this synthetic data to learn robust, class-discriminative representations.
  arXiv  Detail & Related papers  (2025-11-24T19:34:53Z)
• MEG-GPT: A transformer-based foundation model for magnetoencephalography data [6.336623115095147]
  Recent advances in deep learning have enabled significant progress in other domains, such as language and vision, by using foundation models at scale.<n>Here, we introduce MEG-GPT, a transformer based foundation model that uses time-attention and next time-point prediction.<n>We trained MEG-GPT on tokenised brain region time-courses extracted from a large-scale MEG dataset.
  arXiv  Detail & Related papers  (2025-10-20T20:18:38Z)
• Unsupervised Multi-Attention Meta Transformer for Rotating Machinery Fault Diagnosis [4.9825074884178955]
  We propose a Multi-Attention Meta Transformer method for few-shot unsupervised rotating machinery fault diagnosis (MMT-FD)<n>This framework extracts potential fault representations from unlabeled data and demonstrates strong generalization capabilities.<n>The model is iteratively optimized using a small number of contrastive learning iterations, resulting in high efficiency.
  arXiv  Detail & Related papers  (2025-09-11T08:35:43Z)
• Consistency-based Abductive Reasoning over Perceptual Errors of Multiple Pre-trained Models in Novel Environments [5.5855749614100825]
  This paper addresses the hypothesis that leveraging multiple pre-trained models can mitigate this recall reduction.<n>We formulate the challenge of identifying and managing conflicting predictions from various models as a consistency-based abduction problem.<n>Our results validate the use of consistency-based abduction as an effective mechanism to robustly integrate knowledge from multiple imperfect models in challenging, novel scenarios.
  arXiv  Detail & Related papers  (2025-05-25T23:17:47Z)
• AdvKT: An Adversarial Multi-Step Training Framework for Knowledge Tracing [64.79967583649407]
  Knowledge Tracing (KT) monitors students' knowledge states and simulates their responses to question sequences. Existing KT models typically follow a single-step training paradigm, which leads to significant error accumulation. We propose a novel Adversarial Multi-Step Training Framework for Knowledge Tracing (AdvKT) which focuses on the multi-step KT task.
  arXiv  Detail & Related papers  (2025-04-07T03:31:57Z)
• STAR: A Foundation Model-driven Framework for Robust Task Planning and Failure Recovery in Robotic Systems [5.426894918217948]
  STAR (Smart Task Adaptation and Recovery) is a novel framework that synergizes Foundation Models (FMs) with dynamically expanding Knowledge Graphs (KGs) FMs offer remarkable generalization and contextual reasoning, but their limitations hinder reliable deployment. We show that STAR demonstrated an 86% task planning accuracy and 78% recovery success rate, showing significant improvements over baseline methods.
  arXiv  Detail & Related papers  (2025-03-08T05:05:21Z)
• A Predictive Approach for Enhancing Accuracy in Remote Robotic Surgery Using Informer Model [0.0]
  This paper presents a prediction model based on the Transformer-based Informer framework for accurate and efficient position estimation. A comparison with models such as TCN, RNN, and LSTM demonstrates the Informer framework's superior performance in handling position prediction.
  arXiv  Detail & Related papers  (2025-01-24T17:57:00Z)
• Model Predictive Task Sampling for Efficient and Robust Adaptation [46.92143725900031]
  We introduce Model Predictive Task Sampling (MPTS), a framework that bridges the task space and adaptation risk landscape. MPTS employs a generative model to characterize the episodic optimization process and predicts task-specific adaptation risk via posterior inference. MPTS seamlessly integrates into zero-shot, few-shot, and supervised finetuning settings.
  arXiv  Detail & Related papers  (2025-01-19T13:14:53Z)
• Self-Supervised Pre-training Tasks for an fMRI Time-series Transformer in Autism Detection [3.665816629105171]
  Autism Spectrum Disorder (ASD) is a neurodevelopmental condition that encompasses a wide variety of symptoms and degrees of impairment. We have developed a transformer-based self-supervised framework that directly analyzes time-series fMRI data without computing functional connectivity. We show that randomly masking entire ROIs gives better model performance than randomly masking time points in the pre-training step.
  arXiv  Detail & Related papers  (2024-09-18T20:29:23Z)
• Open-Set Deepfake Detection: A Parameter-Efficient Adaptation Method with Forgery Style Mixture [58.60915132222421]
  We introduce an approach that is both general and parameter-efficient for face forgery detection. We design a forgery-style mixture formulation that augments the diversity of forgery source domains. We show that the designed model achieves state-of-the-art generalizability with significantly reduced trainable parameters.
  arXiv  Detail & Related papers  (2024-08-23T01:53:36Z)
• Enhancing Microgrid Performance Prediction with Attention-based Deep Learning Models [0.0]
  This research aims to address microgrid systems' operational challenges, characterized by power oscillations that contribute to grid instability. An integrated strategy is proposed, leveraging the strengths of convolutional and Gated Recurrent Unit (GRU) layers. The framework is anchored by a Multi-Layer Perceptron (MLP) model, which is tasked with comprehensive load forecasting.
  arXiv  Detail & Related papers  (2024-07-20T21:24:11Z)
• Task-Distributionally Robust Data-Free Meta-Learning [99.56612787882334]
  Data-Free Meta-Learning (DFML) aims to efficiently learn new tasks by leveraging multiple pre-trained models without requiring their original training data. For the first time, we reveal two major challenges hindering their practical deployments: Task-Distribution Shift ( TDS) and Task-Distribution Corruption (TDC)
  arXiv  Detail & Related papers  (2023-11-23T15:46:54Z)
• Causal Disentanglement Hidden Markov Model for Fault Diagnosis [55.90917958154425]
  We propose a Causal Disentanglement Hidden Markov model (CDHM) to learn the causality in the bearing fault mechanism. Specifically, we make full use of the time-series data and progressively disentangle the vibration signal into fault-relevant and fault-irrelevant factors. To expand the scope of the application, we adopt unsupervised domain adaptation to transfer the learned disentangled representations to other working environments.
  arXiv  Detail & Related papers  (2023-08-06T05:58:45Z)
• T4PdM: a Deep Neural Network based on the Transformer Architecture for Fault Diagnosis of Rotating Machinery [0.0]
  This paper develops an automatic fault classifier model for predictive maintenance based on a modified version of the Transformer architecture, namely T4PdM. T4PdM was able to achieve an overall accuracy of 99.98% and 98% for both datasets. It has demonstrated the superiority of the model in detecting and classifying faults in rotating industrial machinery.
  arXiv  Detail & Related papers  (2022-04-07T20:31:45Z)
• SUPERB-SG: Enhanced Speech processing Universal PERformance Benchmark for Semantic and Generative Capabilities [76.97949110580703]
  We introduce SUPERB-SG, a new benchmark to evaluate pre-trained models across various speech tasks. We use a lightweight methodology to test the robustness of representations learned by pre-trained models under shifts in data domain. We also show that the task diversity of SUPERB-SG coupled with limited task supervision is an effective recipe for evaluating the generalizability of model representation.
  arXiv  Detail & Related papers  (2022-03-14T04:26:40Z)
• Leveraging Unlabeled Data to Predict Out-of-Distribution Performance [63.740181251997306]
  Real-world machine learning deployments are characterized by mismatches between the source (training) and target (test) distributions. In this work, we investigate methods for predicting the target domain accuracy using only labeled source data and unlabeled target data. We propose Average Thresholded Confidence (ATC), a practical method that learns a threshold on the model's confidence, predicting accuracy as the fraction of unlabeled examples.
  arXiv  Detail & Related papers  (2022-01-11T23:01:12Z)
• ALT-MAS: A Data-Efficient Framework for Active Testing of Machine Learning Algorithms [58.684954492439424]
  We propose a novel framework to efficiently test a machine learning model using only a small amount of labeled test data. The idea is to estimate the metrics of interest for a model-under-test using Bayesian neural network (BNN)
  arXiv  Detail & Related papers  (2021-04-11T12:14:04Z)
• On Fast Adversarial Robustness Adaptation in Model-Agnostic Meta-Learning [100.14809391594109]
  Model-agnostic meta-learning (MAML) has emerged as one of the most successful meta-learning techniques in few-shot learning. Despite the generalization power of the meta-model, it remains elusive that how adversarial robustness can be maintained by MAML in few-shot learning. We propose a general but easily-optimized robustness-regularized meta-learning framework, which allows the use of unlabeled data augmentation, fast adversarial attack generation, and computationally-light fine-tuning.
  arXiv  Detail & Related papers  (2021-02-20T22:03:04Z)
• Reinforcement Learning with Probabilistic Boolean Network Models of Smart Grid Devices [0.3867363075280544]
  We show-case the application of a complex-adaptive, self-organizing modeling method, Probabilistic Boolean Networks (PBN) This work demonstrates that PBNs are is equivalent to the standard Reinforcement Learning Cycle, in which the agent/model has an inter-action with its environment and receives feedback from it in the form of a reward signal.
  arXiv  Detail & Related papers  (2021-02-02T04:13:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.