BatteryBERT for Realistic Battery Fault Detection Using Point-Masked Signal Modeling

• URL: http://arxiv.org/abs/2506.15712v1
• Date: Sat, 31 May 2025 06:06:08 GMT
• Title: BatteryBERT for Realistic Battery Fault Detection Using Point-Masked Signal Modeling
• Authors: Songqi Zhou, Ruixue Liu, Yixing Wang, Jia Lu, Benben Jiang,
• Abstract summary: We propose a novel framework that adapts BERT-style pretraining for battery fault detection.<n>We extend the standard BERT architecture with a customized time-series-to-token representation module and a point-level Masked Signal Modeling (point-MSM) pretraining task.<n>This approach enables self-supervised learning on sequential current, voltage and other charge-discharge cycle data.
• Score: 1.7397173676239939
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Accurate fault detection in lithium-ion batteries is essential for the safe and reliable operation of electric vehicles and energy storage systems. However, existing methods often struggle to capture complex temporal dependencies and cannot fully leverage abundant unlabeled data. Although large language models (LLMs) exhibit strong representation capabilities, their architectures are not directly suited to the numerical time-series data common in industrial settings. To address these challenges, we propose a novel framework that adapts BERT-style pretraining for battery fault detection by extending the standard BERT architecture with a customized time-series-to-token representation module and a point-level Masked Signal Modeling (point-MSM) pretraining task tailored to battery applications. This approach enables self-supervised learning on sequential current, voltage, and other charge-discharge cycle data, yielding distributionally robust, context-aware temporal embeddings. We then concatenate these embeddings with battery metadata and feed them into a downstream classifier for accurate fault classification. Experimental results on a large-scale real-world dataset show that models initialized with our pretrained parameters significantly improve both representation quality and classification accuracy, achieving an AUROC of 0.945 and substantially outperforming existing approaches. These findings validate the effectiveness of BERT-style pretraining for time-series fault detection.

Related papers

• State of Health Estimation of Batteries Using a Time-Informed Dynamic Sequence-Inverted Transformer [0.0]
  Batteries play a central role in the efficiency and safety of energy storage systems, yet they inevitably degrade over time due to repeated charge-discharge cycles.<n> Accurate estimation of a State of Health (SoH) of battery is therefore essential for ensuring operational reliability and safety.<n>We propose a novel architecture: Time-Informed Dynamic Sequence Inverted Transformer (TIDSIT)<n> Experimental results on the NASA battery degradation dataset show that TIDSIT significantly outperforms existing models, achieving over 50% reduction in prediction error and maintaining an SoH prediction error below 0.58%.
  arXiv  Detail & Related papers  (2025-07-24T11:43:46Z)
• End-to-End Framework for Predicting the Remaining Useful Life of Lithium-Ion Batteries [0.04660328753262073]
  This paper proposes a RUL prediction approach that leverages data from recent charge-discharge cycles to estimate the number of remaining usable cycles.<n>The approach introduces both a novel signal processing pipeline and a deep learning prediction model.<n> Experimental results on two publicly available large-scale datasets demonstrate that the proposed method outperforms a baseline deep learning approach and machine learning techniques.
  arXiv  Detail & Related papers  (2025-05-22T13:28:18Z)
• AdvKT: An Adversarial Multi-Step Training Framework for Knowledge Tracing [64.79967583649407]
  Knowledge Tracing (KT) monitors students' knowledge states and simulates their responses to question sequences.<n>Existing KT models typically follow a single-step training paradigm, which leads to significant error accumulation.<n>We propose a novel Adversarial Multi-Step Training Framework for Knowledge Tracing (AdvKT) which focuses on the multi-step KT task.
  arXiv  Detail & Related papers  (2025-04-07T03:31:57Z)
• Powerformer: A Transformer with Weighted Causal Attention for Time-series Forecasting [50.298817606660826]
  We introduce Powerformer, a novel Transformer variant that replaces noncausal attention weights with causal weights that are reweighted according to a smooth heavy-tailed decay.<n>Our empirical results demonstrate that Powerformer achieves state-of-the-art accuracy on public time-series benchmarks.<n>Our analyses show that the model's locality bias is amplified during training, demonstrating an interplay between time-series data and power-law-based attention.
  arXiv  Detail & Related papers  (2025-02-10T04:42:11Z)
• PeFAD: A Parameter-Efficient Federated Framework for Time Series Anomaly Detection [51.20479454379662]
  We propose a. Federated Anomaly Detection framework named PeFAD with the increasing privacy concerns. We conduct extensive evaluations on four real datasets, where PeFAD outperforms existing state-of-the-art baselines by up to 28.74%.
  arXiv  Detail & Related papers  (2024-06-04T13:51:08Z)
• Towards Long-Term Time-Series Forecasting: Feature, Pattern, and Distribution [57.71199089609161]
  Long-term time-series forecasting (LTTF) has become a pressing demand in many applications, such as wind power supply planning. Transformer models have been adopted to deliver high prediction capacity because of the high computational self-attention mechanism. We propose an efficient Transformerbased model, named Conformer, which differentiates itself from existing methods for LTTF in three aspects.
  arXiv  Detail & Related papers  (2023-01-05T13:59:29Z)
• A Robust and Explainable Data-Driven Anomaly Detection Approach For Power Electronics [56.86150790999639]
  We present two anomaly detection and classification approaches, namely the Matrix Profile algorithm and anomaly transformer. The Matrix Profile algorithm is shown to be well suited as a generalizable approach for detecting real-time anomalies in streaming time-series data. A series of custom filters is created and added to the detector to tune its sensitivity, recall, and detection accuracy.
  arXiv  Detail & Related papers  (2022-09-23T06:09:35Z)
• Ranking-Based Physics-Informed Line Failure Detection in Power Grids [66.0797334582536]
  Real-time and accurate detecting of potential line failures is the first step to mitigating the extreme weather impact and activating emergency controls. Power balance equations nonlinearity, increased uncertainty in generation during extreme events, and lack of grid observability compromise the efficiency of traditional data-driven failure detection methods. This paper proposes a Physics-InformEd Line failure Detector (FIELD) that leverages grid topology information to reduce sample and time complexities and improve localization accuracy.
  arXiv  Detail & Related papers  (2022-08-31T18:19:25Z)
• Dynaformer: A Deep Learning Model for Ageing-aware Battery Discharge Prediction [2.670887944566458]
  We introduce a novel Transformer-based deep learning architecture which is able to simultaneously infer the ageing state from a limited number of voltage/current samples. Our experiments show that the trained model is effective for input current profiles of different complexities and is robust to a wide range of degradation levels.
  arXiv  Detail & Related papers  (2022-06-01T15:31:06Z)
• Data-driven Residual Generation for Early Fault Detection with Limited Data [4.129225533930966]
  In many complex systems it is not feasible to develop highly accurate models for the systems. Data-driven solutions have received an immense attention in the industries systems for several practical reasons. Unlike the model-based methods it is straight forward to combine time series measurements such as pressure and voltage with other sources of information.
  arXiv  Detail & Related papers  (2021-09-28T03:18:03Z)
• Robust Data-Driven Error Compensation for a Battery Model [0.0]
  Today's massively collected battery data is not yet used for more accurate and reliable simulations. A data-driven error model is introduced enhancing an existing physically motivated model. A neural network compensates the existing dynamic error and is further limited based on a description of the underlying data.
  arXiv  Detail & Related papers  (2020-12-31T16:11:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.