Maximizing Model Generalization for Machine Condition Monitoring with Self-Supervised Learning and Federated Learning

• URL: http://arxiv.org/abs/2304.14398v2
• Date: Fri, 22 Sep 2023 00:44:44 GMT
• Title: Maximizing Model Generalization for Machine Condition Monitoring with Self-Supervised Learning and Federated Learning
• Authors: Matthew Russell and Peng Wang
• Abstract summary: Deep Learning can diagnose faults and assess machine health from raw condition monitoring data without manually designed statistical features. Traditional supervised learning may struggle to learn compact, discriminative representations that generalize to unseen target domains. This study proposes focusing on maximizing the feature generality on the source domain and applying TL via weight transfer to copy the model to the target domain.
• Score: 4.214064911004321
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep Learning (DL) can diagnose faults and assess machine health from raw condition monitoring data without manually designed statistical features. However, practical manufacturing applications remain extremely difficult for existing DL methods. Machine data is often unlabeled and from very few health conditions (e.g., only normal operating data). Furthermore, models often encounter shifts in domain as process parameters change and new categories of faults emerge. Traditional supervised learning may struggle to learn compact, discriminative representations that generalize to these unseen target domains since it depends on having plentiful classes to partition the feature space with decision boundaries. Transfer Learning (TL) with domain adaptation attempts to adapt these models to unlabeled target domains but assumes similar underlying structure that may not be present if new faults emerge. This study proposes focusing on maximizing the feature generality on the source domain and applying TL via weight transfer to copy the model to the target domain. Specifically, Self-Supervised Learning (SSL) with Barlow Twins may produce more discriminative features for monitoring health condition than supervised learning by focusing on semantic properties of the data. Furthermore, Federated Learning (FL) for distributed training may also improve generalization by efficiently expanding the effective size and diversity of training data by sharing information across multiple client machines. Results show that Barlow Twins outperforms supervised learning in an unlabeled target domain with emerging motor faults when the source training data contains very few distinct categories. Incorporating FL may also provide a slight advantage by diffusing knowledge of health conditions between machines.

Related papers

• Transfer Learning with Clinical Concept Embeddings from Large Language Models [4.838020198075334]
  Large Language Models (LLMs) show significant potential of capturing the semantic meaning of clinical concepts. This study analyzed electronic health records from two large healthcare systems to assess the impact of semantic embeddings on local, shared, and transfer learning models.
  arXiv  Detail & Related papers  (2024-09-20T20:50:55Z)
• Learning Invariant Molecular Representation in Latent Discrete Space [52.13724532622099]
  We propose a new framework for learning molecular representations that exhibit invariance and robustness against distribution shifts. Our model achieves stronger generalization against state-of-the-art baselines in the presence of various distribution shifts.
  arXiv  Detail & Related papers  (2023-10-22T04:06:44Z)
• Unifying and Personalizing Weakly-supervised Federated Medical Image Segmentation via Adaptive Representation and Aggregation [1.121358474059223]
  Federated learning (FL) enables multiple sites to collaboratively train powerful deep models without compromising data privacy and security. Weakly supervised segmentation, which uses sparsely-grained supervision, is increasingly being paid attention to due to its great potential of reducing annotation costs. We propose a novel personalized FL framework for medical image segmentation, named FedICRA, which uniformly leverages heterogeneous weak supervision.
  arXiv  Detail & Related papers  (2023-04-12T06:32:08Z)
• Learning the Finer Things: Bayesian Structure Learning at the Instantiation Level [0.0]
  Successful machine learning methods require a trade-off between memorization and generalization. We present a novel probabilistic graphical model structure learning approach that can learn, generalize and explain in elusive domains.
  arXiv  Detail & Related papers  (2023-03-08T02:31:49Z)
• Robustness, Evaluation and Adaptation of Machine Learning Models in the Wild [4.304803366354879]
  We study causes of impaired robustness to domain shifts and present algorithms for training domain robust models. A key source of model brittleness is due to domain overfitting, which our new training algorithms suppress and instead encourage domain-general hypotheses.
  arXiv  Detail & Related papers  (2023-03-05T21:41:16Z)
• CHALLENGER: Training with Attribution Maps [63.736435657236505]
  We show that utilizing attribution maps for training neural networks can improve regularization of models and thus increase performance. In particular, we show that our generic domain-independent approach yields state-of-the-art results in vision, natural language processing and on time series tasks.
  arXiv  Detail & Related papers  (2022-05-30T13:34:46Z)
• Deep face recognition with clustering based domain adaptation [57.29464116557734]
  We propose a new clustering-based domain adaptation method designed for face recognition task in which the source and target domain do not share any classes. Our method effectively learns the discriminative target feature by aligning the feature domain globally, and, at the meantime, distinguishing the target clusters locally.
  arXiv  Detail & Related papers  (2022-05-27T12:29:11Z)
• Forget Less, Count Better: A Domain-Incremental Self-Distillation Learning Benchmark for Lifelong Crowd Counting [51.44987756859706]
  Off-the-shelf methods have some drawbacks to handle multiple domains. Lifelong Crowd Counting aims at alleviating the catastrophic forgetting and improving the generalization ability.
  arXiv  Detail & Related papers  (2022-05-06T15:37:56Z)
• Federated Contrastive Learning for Volumetric Medical Image Segmentation [16.3860181959878]
  Federated learning (FL) can help in this regard by learning a shared model while keeping training data local for privacy. Traditional FL requires fully-labeled data for training, which is inconvenient or sometimes infeasible to obtain. In this work, we propose an FCL framework for volumetric medical image segmentation with limited annotations.
  arXiv  Detail & Related papers  (2022-04-23T03:47:23Z)
• Adaptive Risk Minimization: Learning to Adapt to Domain Shift [109.87561509436016]
  A fundamental assumption of most machine learning algorithms is that the training and test data are drawn from the same underlying distribution. In this work, we consider the problem setting of domain generalization, where the training data are structured into domains and there may be multiple test time shifts. We introduce the framework of adaptive risk minimization (ARM), in which models are directly optimized for effective adaptation to shift by learning to adapt on the training domains.
  arXiv  Detail & Related papers  (2020-07-06T17:59:30Z)
• Few-Shot Learning as Domain Adaptation: Algorithm and Analysis [120.75020271706978]
  Few-shot learning uses prior knowledge learned from the seen classes to recognize the unseen classes. This class-difference-caused distribution shift can be considered as a special case of domain shift. We propose a prototypical domain adaptation network with attention (DAPNA) to explicitly tackle such a domain shift problem in a meta-learning framework.
  arXiv  Detail & Related papers  (2020-02-06T01:04:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.