Remaining Useful Life Estimation of Hard Disk Drives using Bidirectional LSTM Networks

• URL: http://arxiv.org/abs/2109.05351v1
• Date: Sat, 11 Sep 2021 19:26:07 GMT
• Title: Remaining Useful Life Estimation of Hard Disk Drives using Bidirectional LSTM Networks
• Authors: Austin Coursey, Gopal Nath, Srikanth Prabhu and Saptarshi Sengupta
• Abstract summary: We introduce methods of extracting meaningful attributes associated with operational failure and of pre-processing health statistics data. We use a Bidirectional LSTM with a multi-day look back period to learn the temporal progression of health indicators and baseline them against vanilla LSTM and Random Forest models. Our approach can predict the occurrence of disk failure with an accuracy of 96.4% considering test data 60 days before failure.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Physical and cloud storage services are well-served by functioning and reliable high-volume storage systems. Recent observations point to hard disk reliability as one of the most pressing reliability issues in data centers containing massive volumes of storage devices such as HDDs. In this regard, early detection of impending failure at the disk level aids in reducing system downtime and reduces operational loss making proactive health monitoring a priority for AIOps in such settings. In this work, we introduce methods of extracting meaningful attributes associated with operational failure and of pre-processing the highly imbalanced health statistics data for subsequent prediction tasks using data-driven approaches. We use a Bidirectional LSTM with a multi-day look back period to learn the temporal progression of health indicators and baseline them against vanilla LSTM and Random Forest models to come up with several key metrics that establish the usefulness of and superiority of our model under some tightly defined operational constraints. For example, using a 15 day look back period, our approach can predict the occurrence of disk failure with an accuracy of 96.4% considering test data 60 days before failure. This helps to alert operations maintenance well in-advance about potential mitigation needs. In addition, our model reports a mean absolute error of 0.12 for predicting failure up to 60 days in advance, placing it among the state-of-the-art in recent literature.

Related papers

• On the Generalizability of Machine Learning-based Ransomware Detection in Block Storage [0.0]
  We propose a kernel-based framework capable of efficiently extracting and analyzing IO operations to identify ransomware activity. Our method employs a refined set of computationally light features optimized for ML models to accurately discern malicious from benign activities. Empirical validation reveals that our decision tree-based models achieve remarkable effectiveness.
  arXiv  Detail & Related papers  (2024-12-30T17:02:37Z)
• Recall-driven Precision Refinement: Unveiling Accurate Fall Detection using LSTM [0.0]
  This paper presents an innovative approach to address the pressing concern of fall incidents among the elderly by developing an accurate fall detection system. Our proposed system combines state-of-the-art technologies, including accelerometer and gyroscope sensors, with deep learning models, specifically Long Short-Term Memory (LSTM) networks. We introduce pruning techniques that strategically fine-tune the LSTM model's architecture and parameters to optimize the system's performance.
  arXiv  Detail & Related papers  (2023-09-09T20:17:39Z)
• TFBEST: Dual-Aspect Transformer with Learnable Positional Encoding for Failure Prediction [1.223779595809275]
  We propose a Temporal-fusion Bi-encoder Self-attention Transformer (TFBEST) for predicting failures in hard-drives. It is an encoder-decoder based deep learning technique that enhances the context gained from understanding health statistics. Experiments on HDD data show that our method significantly outperforms the state-of-the-art RUL prediction methods.
  arXiv  Detail & Related papers  (2023-09-06T01:03:14Z)
• Enterprise Disk Drive Scrubbing Based on Mondrian Conformal Predictors [1.290382979353427]
  scrubbing the entire storage array at once can adversely impact system performance. We propose a selective disk scrubbing method that enhances the overall reliability and power efficiency in data centers. By scrubbing just 22.7% of the total storage disks, we can achieve optimized energy consumption and reduce the carbon footprint of the data center.
  arXiv  Detail & Related papers  (2023-06-01T04:11:22Z)
• Large-scale End-of-Life Prediction of Hard Disks in Distributed Datacenters [0.0]
  Large-scale predictive analyses are performed using severely skewed health statistics data. We present an encoder-decoder LSTM model where the context gained from understanding health statistics sequences aid in predicting an output sequence of the number of days remaining before a disk potentially fails.
  arXiv  Detail & Related papers  (2023-03-15T21:55:07Z)
• PULL: Reactive Log Anomaly Detection Based On Iterative PU Learning [58.85063149619348]
  We propose PULL, an iterative log analysis method for reactive anomaly detection based on estimated failure time windows. Our evaluation shows that PULL consistently outperforms ten benchmark baselines across three different datasets.
  arXiv  Detail & Related papers  (2023-01-25T16:34:43Z)
• ARLIF-IDS -- Attention augmented Real-Time Isolation Forest Intrusion Detection System [0.0]
  Internet of Things and Software Defined Networking leverage lightweight strategies for the early detection of DDoS attacks. It is essential to have a fast and effective security identification model based on low number of features. In this work, a novel Attention-based Isolation Forest Intrusion Detection System is proposed.
  arXiv  Detail & Related papers  (2022-04-20T18:40:23Z)
• Scale-Equivalent Distillation for Semi-Supervised Object Detection [57.59525453301374]
  Recent Semi-Supervised Object Detection (SS-OD) methods are mainly based on self-training, generating hard pseudo-labels by a teacher model on unlabeled data as supervisory signals. We analyze the challenges these methods meet with the empirical experiment results. We introduce a novel approach, Scale-Equivalent Distillation (SED), which is a simple yet effective end-to-end knowledge distillation framework robust to large object size variance and class imbalance.
  arXiv  Detail & Related papers  (2022-03-23T07:33:37Z)
• LogLAB: Attention-Based Labeling of Log Data Anomalies via Weak Supervision [63.08516384181491]
  We present LogLAB, a novel modeling approach for automated labeling of log messages without requiring manual work by experts. Our method relies on estimated failure time windows provided by monitoring systems to produce precise labeled datasets in retrospect. Our evaluation shows that LogLAB consistently outperforms nine benchmark approaches across three different datasets and maintains an F1-score of more than 0.98 even at large failure time windows.
  arXiv  Detail & Related papers  (2021-11-02T15:16:08Z)
• Cloud Failure Prediction with Hierarchical Temporary Memory: An Empirical Assessment [64.73243241568555]
  Hierarchical Temporary Memory (HTM) is an unsupervised learning algorithm inspired by the features of the neocortex. This paper presents the first systematic study that assesses HTM in the context of failure prediction.
  arXiv  Detail & Related papers  (2021-10-06T07:09:45Z)
• Predicting Parkinson's Disease with Multimodal Irregularly Collected Longitudinal Smartphone Data [75.23250968928578]
  Parkinsons Disease is a neurological disorder and prevalent in elderly people. Traditional ways to diagnose the disease rely on in-person subjective clinical evaluations on the quality of a set of activity tests. We propose a novel time-series based approach to predicting Parkinson's Disease with raw activity test data collected by smartphones in the wild.
  arXiv  Detail & Related papers  (2020-09-25T01:50:15Z)
• TadGAN: Time Series Anomaly Detection Using Generative Adversarial Networks [73.01104041298031]
  TadGAN is an unsupervised anomaly detection approach built on Generative Adversarial Networks (GANs) To capture the temporal correlations of time series, we use LSTM Recurrent Neural Networks as base models for Generators and Critics. To demonstrate the performance and generalizability of our approach, we test several anomaly scoring techniques and report the best-suited one.
  arXiv  Detail & Related papers  (2020-09-16T15:52:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.