Attention-embedded Quadratic Network (Qttention) for Effective and Interpretable Bearing Fault Diagnosis

• URL: http://arxiv.org/abs/2206.00390v1
• Date: Wed, 1 Jun 2022 10:51:01 GMT
• Title: Attention-embedded Quadratic Network (Qttention) for Effective and Interpretable Bearing Fault Diagnosis
• Authors: Jing-Xiao Liao, Hang-Cheng Dong, Zhi-Qi Sun, Jinwei Sun, Shiping Zhang, Feng-Lei Fan
• Abstract summary: Bearing fault diagnosis is of great importance to decrease the damage risk of rotating machines and further improve economic profits. Recently, machine learning, represented by deep learning, has made great progress in bearing fault diagnosis. Applying deep learning to such a task still faces two major problems.
• Score: 0.31317409221921144
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Bearing fault diagnosis is of great importance to decrease the damage risk of rotating machines and further improve economic profits. Recently, machine learning, represented by deep learning, has made great progress in bearing fault diagnosis. However, applying deep learning to such a task still faces two major problems. On the one hand, deep learning loses its effectiveness when bearing data are noisy or big data are unavailable, making deep learning hard to implement in industrial fields. On the other hand, a deep network is notoriously a black box. It is difficult to know how a model classifies faulty signals from the normal and the physics principle behind the classification. To solve the effectiveness and interpretability issues, we prototype a convolutional network with recently-invented quadratic neurons. This quadratic neuron empowered network can qualify the noisy and small bearing data due to the strong feature representation ability of quadratic neurons. Moreover, we independently derive the attention mechanism from a quadratic neuron, referred to as qttention, by factorizing the learned quadratic function in analogue to the attention, making the model with quadratic neurons inherently interpretable. Experiments on the public and our datasets demonstrate that the proposed network can facilitate effective and interpretable bearing fault diagnosis.

Related papers

• Coding schemes in neural networks learning classification tasks [52.22978725954347]
  We investigate fully-connected, wide neural networks learning classification tasks. We show that the networks acquire strong, data-dependent features. Surprisingly, the nature of the internal representations depends crucially on the neuronal nonlinearity.
  arXiv  Detail & Related papers  (2024-06-24T14:50:05Z)
• Redundancy and Concept Analysis for Code-trained Language Models [5.726842555987591]
  Code-trained language models have proven to be highly effective for various code intelligence tasks. They can be challenging to train and deploy for many software engineering applications due to computational bottlenecks and memory constraints. We perform the first neuron-level analysis for source code models to identify textitimportant neurons within latent representations.
  arXiv  Detail & Related papers  (2023-05-01T15:22:41Z)
• One Neuron Saved Is One Neuron Earned: On Parametric Efficiency of Quadratic Networks [21.5187335186035]
  We show that quadratic networks enjoy parametric efficiency, thereby confirming that the superior performance of quadratic networks is due to the intrinsic expressive capability. From the perspective of the Barron space, we demonstrate that there exists a functional space whose functions can be approximated by quadratic networks in a dimension-free error.
  arXiv  Detail & Related papers  (2023-03-11T05:32:18Z)
• An Adaptive Contrastive Learning Model for Spike Sorting [12.043679000694258]
  In neuroscience research, it is important to separate out the activity of individual neurons. With the development of large-scale silicon technology, artificially interpreting and labeling spikes is becoming increasingly impractical. We propose a novel modeling framework that learns representations from spikes through contrastive learning.
  arXiv  Detail & Related papers  (2022-05-24T09:18:46Z)
• Causal Discovery and Knowledge Injection for Contestable Neural Networks (with Appendices) [10.616061367794385]
  We propose a two-way interaction whereby neural-network-empowered machines can expose the underpinning learnt causal graphs. We show that our method improves predictive performance up to 2.4x while producing parsimonious networks, up to 7x smaller in the input layer.
  arXiv  Detail & Related papers  (2022-05-19T18:21:12Z)
• On Expressivity and Trainability of Quadratic Networks [12.878230964137014]
  quadratic artificial neurons can play an important role in deep learning models. We show that the superior expressivity of a quadratic network over either a conventional network or a conventional network via quadratic activation is not fully elucidated. We propose an effective training strategy referred to as ReLinear to stabilize the training process of a quadratic network.
  arXiv  Detail & Related papers  (2021-10-12T15:33:32Z)
• The Causal Neural Connection: Expressiveness, Learnability, and Inference [125.57815987218756]
  An object called structural causal model (SCM) represents a collection of mechanisms and sources of random variation of the system under investigation. In this paper, we show that the causal hierarchy theorem (Thm. 1, Bareinboim et al., 2020) still holds for neural models. We introduce a special type of SCM called a neural causal model (NCM), and formalize a new type of inductive bias to encode structural constraints necessary for performing causal inferences.
  arXiv  Detail & Related papers  (2021-07-02T01:55:18Z)
• A neural anisotropic view of underspecification in deep learning [60.119023683371736]
  We show that the way neural networks handle the underspecification of problems is highly dependent on the data representation. Our results highlight that understanding the architectural inductive bias in deep learning is fundamental to address the fairness, robustness, and generalization of these systems.
  arXiv  Detail & Related papers  (2021-04-29T14:31:09Z)
• Artificial Neural Variability for Deep Learning: On Overfitting, Noise Memorization, and Catastrophic Forgetting [135.0863818867184]
  artificial neural variability (ANV) helps artificial neural networks learn some advantages from natural'' neural networks. ANV plays as an implicit regularizer of the mutual information between the training data and the learned model. It can effectively relieve overfitting, label noise memorization, and catastrophic forgetting at negligible costs.
  arXiv  Detail & Related papers  (2020-11-12T06:06:33Z)
• Vulnerability Under Adversarial Machine Learning: Bias or Variance? [77.30759061082085]
  We investigate the effect of adversarial machine learning on the bias and variance of a trained deep neural network. Our analysis sheds light on why the deep neural networks have poor performance under adversarial perturbation. We introduce a new adversarial machine learning algorithm with lower computational complexity than well-known adversarial machine learning strategies.
  arXiv  Detail & Related papers  (2020-08-01T00:58:54Z)
• Towards Efficient Processing and Learning with Spikes: New Approaches for Multi-Spike Learning [59.249322621035056]
  We propose two new multi-spike learning rules which demonstrate better performance over other baselines on various tasks. In the feature detection task, we re-examine the ability of unsupervised STDP with its limitations being presented. Our proposed learning rules can reliably solve the task over a wide range of conditions without specific constraints being applied.
  arXiv  Detail & Related papers  (2020-05-02T06:41:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.