Supervised Hebbian learning: toward eXplainable AI

• URL: http://arxiv.org/abs/2203.01304v1
• Date: Wed, 2 Mar 2022 18:46:17 GMT
• Title: Supervised Hebbian learning: toward eXplainable AI
• Abstract summary: We prove that, if we feed the Hopfield model with blurred examples, we can define both em supervised and em unsupervised learning protocols. We also prove that, for random, structureless datasets, the Hopfield model equipped with a supervised learning rule is equivalent to a restricted Boltzmann machine.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In neural network's Literature, {\em Hebbian learning} traditionally refers to the procedure by which the Hopfield model and its generalizations {\em store} archetypes (i.e., definite patterns that are experienced just once to form the synaptic matrix). However, the term {\em learning} in Machine Learning refers to the ability of the machine to extract features from the supplied dataset (e.g., made of blurred examples of these archetypes), in order to make its own representation of the unavailable archetypes. Here we prove that, if we feed the Hopfield model with blurred examples, we can define both {\em supervised} and {\em unsupervised} learning protocols by which the network can possibly infer the archetypes and we detect the correct control parameters (including the dataset size and its quality) to depict a phase diagram for the system performance. We also prove that, for random, structureless datasets, the Hopfield model equipped with a supervised learning rule is equivalent to a restricted Boltzmann machine and this suggests an optimal training routine; the robustness of results is also checked numerically for structured datasets. This work contributes to pave a solid way toward eXplainable AI (XAI).

Related papers

• Navigating the Latent Space Dynamics of Neural Models [35.39092540369244]
  We present an alternative interpretation of neural models as dynamical systems acting on the latent manifold.<n>We show that autoencoder models implicitly define a latent vector field on the manifold, derived by iteratively applying the encoding-decoding map.<n>We propose to leverage the vector field as a representation for the network, providing a novel tool to analyze the properties of the model and the data.
  arXiv  Detail & Related papers  (2025-05-28T18:57:41Z)
• Transformers are uninterpretable with myopic methods: a case study with bounded Dyck grammars [36.780346257061495]
  Interpretability methods aim to understand the algorithm implemented by a trained model. We take a critical view of methods that exclusively focus on individual parts of the model.
  arXiv  Detail & Related papers  (2023-12-03T15:34:46Z)
• LLM2Loss: Leveraging Language Models for Explainable Model Diagnostics [5.33024001730262]
  We propose an approach that can provide semantic insights into a model's patterns of failures and biases. We show that an ensemble of such lightweight models can be used to generate insights on the performance of the black-box model.
  arXiv  Detail & Related papers  (2023-05-04T23:54:37Z)
• Hopfield model with planted patterns: a teacher-student self-supervised learning model [0.0]
  We show that it is possible to formulate a teacher-student self-supervised learning problem with Boltzmann machines. We analyze the learning performance by studying the phase diagram in terms of the training set size.
  arXiv  Detail & Related papers  (2023-04-26T17:50:52Z)
• A didactic approach to quantum machine learning with a single qubit [68.8204255655161]
  We focus on the case of learning with a single qubit, using data re-uploading techniques. We implement the different proposed formulations in toy and real-world datasets using the qiskit quantum computing SDK.
  arXiv  Detail & Related papers  (2022-11-23T18:25:32Z)
• CHALLENGER: Training with Attribution Maps [63.736435657236505]
  We show that utilizing attribution maps for training neural networks can improve regularization of models and thus increase performance. In particular, we show that our generic domain-independent approach yields state-of-the-art results in vision, natural language processing and on time series tasks.
  arXiv  Detail & Related papers  (2022-05-30T13:34:46Z)
• Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
  We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs. By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
  arXiv  Detail & Related papers  (2022-02-01T17:11:13Z)
• Graph Kernel Neural Networks [53.91024360329517]
  We propose to use graph kernels, i.e. kernel functions that compute an inner product on graphs, to extend the standard convolution operator to the graph domain. This allows us to define an entirely structural model that does not require computing the embedding of the input graph. Our architecture allows to plug-in any type of graph kernels and has the added benefit of providing some interpretability.
  arXiv  Detail & Related papers  (2021-12-14T14:48:08Z)
• Dynamic Inference with Neural Interpreters [72.90231306252007]
  We present Neural Interpreters, an architecture that factorizes inference in a self-attention network as a system of modules. inputs to the model are routed through a sequence of functions in a way that is end-to-end learned. We show that Neural Interpreters perform on par with the vision transformer using fewer parameters, while being transferrable to a new task in a sample efficient manner.
  arXiv  Detail & Related papers  (2021-10-12T23:22:45Z)
• Multi-Branch Deep Radial Basis Function Networks for Facial Emotion Recognition [80.35852245488043]
  We propose a CNN based architecture enhanced with multiple branches formed by radial basis function (RBF) units. RBF units capture local patterns shared by similar instances using an intermediate representation. We show it is the incorporation of local information what makes the proposed model competitive.
  arXiv  Detail & Related papers  (2021-09-07T21:05:56Z)
• The emergence of a concept in shallow neural networks [0.0]
  We consider restricted Boltzmann machine (RBMs) trained over an unstructured dataset made of blurred copies of definite but unavailable archetypes'' We show that there exists a critical sample size beyond which the RBM can learn archetypes.
  arXiv  Detail & Related papers  (2021-09-01T15:56:38Z)
• Capturing the learning curves of generic features maps for realistic data sets with a teacher-student model [24.679669970832396]
  Teacher-student models provide a powerful framework in which the typical case performance of high-dimensional supervised learning tasks can be studied in closed form. In this setting, labels are assigned to data - often taken to be Gaussian i.i.d. - by a teacher model, and the goal is to characterise the typical performance of the student model in recovering the parameters that generated the labels.
  arXiv  Detail & Related papers  (2021-02-16T12:49:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.