The Life and Death of SSDs and HDDs: Similarities, Differences, and Prediction Models

• URL: http://arxiv.org/abs/2012.12373v1
• Date: Tue, 22 Dec 2020 21:50:32 GMT
• Title: The Life and Death of SSDs and HDDs: Similarities, Differences, and Prediction Models
• Authors: Riccardo Pinciroli, Lishan Yang, Jacob Alter, Evgenia Smirni
• Abstract summary: We present a comparative study of hard disk drives (HDDs) and solid state drives (SSDs) that constitute typical storage in data centers. We characterize the workload conditions that lead to failures and illustrate that their root causes differ from common expectation. We develop several machine learning failure prediction models that are shown to be surprisingly accurate.
• Score: 1.6795461001108098
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Data center downtime typically centers around IT equipment failure. Storage devices are the most frequently failing components in data centers. We present a comparative study of hard disk drives (HDDs) and solid state drives (SSDs) that constitute the typical storage in data centers. Using a six-year field data of 100,000 HDDs of different models from the same manufacturer from the BackBlaze dataset and a six-year field data of 30,000 SSDs of three models from a Google data center, we characterize the workload conditions that lead to failures and illustrate that their root causes differ from common expectation but remain difficult to discern. For the case of HDDs we observe that young and old drives do not present many differences in their failures. Instead, failures may be distinguished by discriminating drives based on the time spent for head positioning. For SSDs, we observe high levels of infant mortality and characterize the differences between infant and non-infant failures. We develop several machine learning failure prediction models that are shown to be surprisingly accurate, achieving high recall and low false positive rates. These models are used beyond simple prediction as they aid us to untangle the complex interaction of workload characteristics that lead to failures and identify failure root causes from monitored symptoms.

Related papers

• Uncertainty Estimation for 3D Object Detection via Evidential Learning [63.61283174146648]
  We introduce a framework for quantifying uncertainty in 3D object detection by leveraging an evidential learning loss on Bird's Eye View representations in the 3D detector. We demonstrate both the efficacy and importance of these uncertainty estimates on identifying out-of-distribution scenes, poorly localized objects, and missing (false negative) detections.
  arXiv  Detail & Related papers  (2024-10-31T13:13:32Z)
• Happy: A Debiased Learning Framework for Continual Generalized Category Discovery [54.54153155039062]
  This paper explores the underexplored task of Continual Generalized Category Discovery (C-GCD) C-GCD aims to incrementally discover new classes from unlabeled data while maintaining the ability to recognize previously learned classes. We introduce a debiased learning framework, namely Happy, characterized by Hardness-aware prototype sampling and soft entropy regularization.
  arXiv  Detail & Related papers  (2024-10-09T04:18:51Z)
• TFBEST: Dual-Aspect Transformer with Learnable Positional Encoding for Failure Prediction [1.223779595809275]
  We propose a Temporal-fusion Bi-encoder Self-attention Transformer (TFBEST) for predicting failures in hard-drives. It is an encoder-decoder based deep learning technique that enhances the context gained from understanding health statistics. Experiments on HDD data show that our method significantly outperforms the state-of-the-art RUL prediction methods.
  arXiv  Detail & Related papers  (2023-09-06T01:03:14Z)
• Performance Modeling of Data Storage Systems using Generative Models [0.5352699766206809]
  We have developed several models of a storage system using machine learning-based generative models. The results of the experiments demonstrate the errors of 4-10 % for IOPS and 3-16 % for latency predictions depending on the components.
  arXiv  Detail & Related papers  (2023-07-05T07:30:53Z)
• Large-scale End-of-Life Prediction of Hard Disks in Distributed Datacenters [0.0]
  Large-scale predictive analyses are performed using severely skewed health statistics data. We present an encoder-decoder LSTM model where the context gained from understanding health statistics sequences aid in predicting an output sequence of the number of days remaining before a disk potentially fails.
  arXiv  Detail & Related papers  (2023-03-15T21:55:07Z)
• Rethinking Bias Mitigation: Fairer Architectures Make for Fairer Face Recognition [107.58227666024791]
  Face recognition systems are widely deployed in safety-critical applications, including law enforcement. They exhibit bias across a range of socio-demographic dimensions, such as gender and race. Previous works on bias mitigation largely focused on pre-processing the training data.
  arXiv  Detail & Related papers  (2022-10-18T15:46:05Z)
• Remaining Useful Life Estimation of Hard Disk Drives using Bidirectional LSTM Networks [0.0]
  We introduce methods of extracting meaningful attributes associated with operational failure and of pre-processing health statistics data. We use a Bidirectional LSTM with a multi-day look back period to learn the temporal progression of health indicators and baseline them against vanilla LSTM and Random Forest models. Our approach can predict the occurrence of disk failure with an accuracy of 96.4% considering test data 60 days before failure.
  arXiv  Detail & Related papers  (2021-09-11T19:26:07Z)
• Semantics-Guided Contrastive Network for Zero-Shot Object detection [67.61512036994458]
  Zero-shot object detection (ZSD) is a new challenge in computer vision. We develop ContrastZSD, a framework that brings contrastive learning mechanism into the realm of zero-shot detection. Our method outperforms the previous state-of-the-art on both ZSD and generalized ZSD tasks.
  arXiv  Detail & Related papers  (2021-09-04T03:32:15Z)
• Discriminative-Generative Dual Memory Video Anomaly Detection [81.09977516403411]
  Recently, people tried to use a few anomalies for video anomaly detection (VAD) instead of only normal data during the training process. We propose a DiscRiminative-gEnerative duAl Memory (DREAM) anomaly detection model to take advantage of a few anomalies and solve data imbalance.
  arXiv  Detail & Related papers  (2021-04-29T15:49:01Z)
• Online detection of failures generated by storage simulator [2.3859858429583665]
  We create a Go-based (golang) package for simulating the behavior of modern storage infrastructure. The package's flexible structure allows us to create a model of a real-world storage system with a number of components. To discover failures in the time series distribution generated by the simulator, we modified a change point detection algorithm that works in online mode.
  arXiv  Detail & Related papers  (2021-01-18T14:56:53Z)
• Stance Detection Benchmark: How Robust Is Your Stance Detection? [65.91772010586605]
  Stance Detection (StD) aims to detect an author's stance towards a certain topic or claim. We introduce a StD benchmark that learns from ten StD datasets of various domains in a multi-dataset learning setting. Within this benchmark setup, we are able to present new state-of-the-art results on five of the datasets.
  arXiv  Detail & Related papers  (2020-01-06T13:37:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.