Disk failure prediction based on multi-layer domain adaptive learning

• URL: http://arxiv.org/abs/2310.06534v1
• Date: Tue, 10 Oct 2023 11:28:40 GMT
• Title: Disk failure prediction based on multi-layer domain adaptive learning
• Authors: Guangfu Gao, Peng Wu and Hussain Dawood
• Abstract summary: This paper presents a novel method for predicting disk failures by leveraging multi-layer domain adaptive learning techniques. It has been demonstrated that the proposed technique can generate a reliable prediction model and improve the ability to predict failures on disk data with few failure samples.
• Score: 6.757777833155211
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Large scale data storage is susceptible to failure. As disks are damaged and replaced, traditional machine learning models, which rely on historical data to make predictions, struggle to accurately predict disk failures. This paper presents a novel method for predicting disk failures by leveraging multi-layer domain adaptive learning techniques. First, disk data with numerous faults is selected as the source domain, and disk data with fewer faults is selected as the target domain. A training of the feature extraction network is performed with the selected origin and destination domains. The contrast between the two domains facilitates the transfer of diagnostic knowledge from the domain of source and target. According to the experimental findings, it has been demonstrated that the proposed technique can generate a reliable prediction model and improve the ability to predict failures on disk data with few failure samples.

Related papers

• Source-Free Test-Time Adaptation For Online Surface-Defect Detection [29.69030283193086]
  We propose a novel test-time adaptation surface-defect detection approach. It adapts pre-trained models to new domains and classes during inference. Experiments demonstrate it outperforms state-of-the-art techniques.
  arXiv  Detail & Related papers  (2024-08-18T14:24:05Z)
• Calibrated Adaptive Teacher for Domain Adaptive Intelligent Fault Diagnosis [7.88657961743755]
  Unsupervised domain adaptation (UDA) deals with the scenario where labeled data are available in a source domain, and only unlabeled data are available in a target domain. We propose a novel UDA method called Calibrated Adaptive Teacher (CAT), where we propose to calibrate the predictions of the teacher network throughout the self-training process.
  arXiv  Detail & Related papers  (2023-12-05T15:19:29Z)
• Bridging Precision and Confidence: A Train-Time Loss for Calibrating Object Detection [58.789823426981044]
  We propose a novel auxiliary loss formulation that aims to align the class confidence of bounding boxes with the accurateness of predictions. Our results reveal that our train-time loss surpasses strong calibration baselines in reducing calibration error for both in and out-domain scenarios.
  arXiv  Detail & Related papers  (2023-03-25T08:56:21Z)
• TransferD2: Automated Defect Detection Approach in Smart Manufacturing using Transfer Learning Techniques [1.8899300124593645]
  We propose a transfer learning approach, namely TransferD2, to correctly identify defects on a dataset of source objects. Our proposed approach can be applied in defect detection applications where insufficient data is available for training a model and can be extended to identify imperfections in new unseen data.
  arXiv  Detail & Related papers  (2023-02-26T13:24:46Z)
• Diversify and Disambiguate: Learning From Underspecified Data [76.67228314592904]
  DivDis is a framework that learns a diverse collection of hypotheses for a task by leveraging unlabeled data from the test distribution. We demonstrate the ability of DivDis to find hypotheses that use robust features in image classification and natural language processing problems with underspecification.
  arXiv  Detail & Related papers  (2022-02-07T18:59:06Z)
• Leveraging Unlabeled Data to Predict Out-of-Distribution Performance [63.740181251997306]
  Real-world machine learning deployments are characterized by mismatches between the source (training) and target (test) distributions. In this work, we investigate methods for predicting the target domain accuracy using only labeled source data and unlabeled target data. We propose Average Thresholded Confidence (ATC), a practical method that learns a threshold on the model's confidence, predicting accuracy as the fraction of unlabeled examples.
  arXiv  Detail & Related papers  (2022-01-11T23:01:12Z)
• SFFDD: Deep Neural Network with Enriched Features for Failure Prediction with Its Application to Computer Disk Driver [0.0]
  We treat the multivariate time series sensor data as images for both visualization and computation. In addition to feature derivation, ensemble method is used to further improve the performance. We apply the proposed method on the early predict failure of computer disk drive in order to improve storage systems availability and avoid data loss.
  arXiv  Detail & Related papers  (2021-09-20T21:43:43Z)
• Distill and Fine-tune: Effective Adaptation from a Black-box Source Model [138.12678159620248]
  Unsupervised domain adaptation (UDA) aims to transfer knowledge in previous related labeled datasets (source) to a new unlabeled dataset (target) We propose a novel two-step adaptation framework called Distill and Fine-tune (Dis-tune)
  arXiv  Detail & Related papers  (2021-04-04T05:29:05Z)
• Robust and Transferable Anomaly Detection in Log Data using Pre-Trained Language Models [59.04636530383049]
  Anomalies or failures in large computer systems, such as the cloud, have an impact on a large number of users. We propose a framework for anomaly detection in log data, as a major troubleshooting source of system information.
  arXiv  Detail & Related papers  (2021-02-23T09:17:05Z)
• $n$-Reference Transfer Learning for Saliency Prediction [73.17061116358036]
  We propose a few-shot transfer learning paradigm for saliency prediction. The proposed framework is gradient-based and model-agnostic. The results show that the proposed framework achieves a significant performance improvement.
  arXiv  Detail & Related papers  (2020-07-09T23:20:44Z)
• DSDANet: Deep Siamese Domain Adaptation Convolutional Neural Network for Cross-domain Change Detection [44.05317423742678]
  We propose a novel deep siamese domain adaptation convolutional neural network architecture for cross-domain change detection. To the best of our knowledge, it is the first time that such a domain adaptation-based deep network is proposed for change detection.
  arXiv  Detail & Related papers  (2020-06-16T15:00:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.