Classifying States of the Hopfield Network with Improved Accuracy, Generalization, and Interpretability

• URL: http://arxiv.org/abs/2503.03018v1
• Date: Tue, 04 Mar 2025 21:29:42 GMT
• Title: Classifying States of the Hopfield Network with Improved Accuracy, Generalization, and Interpretability
• Authors: Hayden McAlister, Anthony Robins, Lech Szymanski,
• Abstract summary: We study the generalizability of different classification models when trained on states derived from different prototype tasks.<n>We find that simple models often outperform the stability ratio while remaining interpretable.
• Score: 1.2289361708127877
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We extend the existing work on Hopfield network state classification, employing more complex models that remain interpretable, such as densely-connected feed-forward deep neural networks and support vector machines. The states of the Hopfield network can be grouped into several classes, including learned (those presented during training), spurious (stable states that were not learned), and prototype (stable states that were not learned but are representative for a subset of learned states). It is often useful to determine to what class a given state belongs to; for example to ignore spurious states when retrieving from the network. Previous research has approached the state classification task with simple linear methods, most notably the stability ratio. We deepen the research on classifying states from prototype-regime Hopfield networks, investigating how varying the factors strengthening prototypes influences the state classification task. We study the generalizability of different classification models when trained on states derived from different prototype tasks -- for example, can a network trained on a Hopfield network with 10 prototypes classify states from a network with 20 prototypes? We find that simple models often outperform the stability ratio while remaining interpretable. These models require surprisingly little training data and generalize exceptionally well to states generated by a range of Hopfield networks, even those that were trained on exceedingly different datasets.

Related papers

• Prototype Analysis in Hopfield Networks with Hebbian Learning [1.2289361708127877]
  Hebbian learning with highly correlated states leads to degraded memory performance. We show this type of learning can lead to prototype formation, where unlearned states emerge as representatives of large correlated subsets of states. This process has similarities to prototype learning in human cognition.
  arXiv  Detail & Related papers  (2024-05-29T01:03:48Z)
• Context-Specific Refinements of Bayesian Network Classifiers [1.9136291802656262]
  We study the relationship between our novel classes of classifiers and Bayesian networks. We introduce and implement data-driven learning routines for our models. The study demonstrates that models embedding asymmetric information can enhance classification accuracy.
  arXiv  Detail & Related papers  (2024-05-28T15:50:50Z)
• A separability-based approach to quantifying generalization: which layer is best? [0.0]
  Generalization to unseen data remains poorly understood for deep learning classification and foundation models. We provide a new method for evaluating the capacity of networks to represent a sampled domain. We find that (i) high classification accuracy does not imply high generalizability; and (ii) deeper layers in a model do not always generalize the best.
  arXiv  Detail & Related papers  (2024-05-02T17:54:35Z)
• Neural Representations Reveal Distinct Modes of Class Fitting in Residual Convolutional Networks [5.1271832547387115]
  We leverage probabilistic models of neural representations to investigate how residual networks fit classes. We find that classes in the investigated models are not fitted in an uniform way. We show that the uncovered structure in neural representations correlate with robustness of training examples and adversarial memorization.
  arXiv  Detail & Related papers  (2022-12-01T18:55:58Z)
• CHALLENGER: Training with Attribution Maps [63.736435657236505]
  We show that utilizing attribution maps for training neural networks can improve regularization of models and thus increase performance. In particular, we show that our generic domain-independent approach yields state-of-the-art results in vision, natural language processing and on time series tasks.
  arXiv  Detail & Related papers  (2022-05-30T13:34:46Z)
• Region Comparison Network for Interpretable Few-shot Image Classification [97.97902360117368]
  Few-shot image classification has been proposed to effectively use only a limited number of labeled examples to train models for new classes. We propose a metric learning based method named Region Comparison Network (RCN), which is able to reveal how few-shot learning works. We also present a new way to generalize the interpretability from the level of tasks to categories.
  arXiv  Detail & Related papers  (2020-09-08T07:29:05Z)
• Hopfield Networks is All You Need [8.508381229662907]
  We introduce a modern Hopfield network with continuous states and a corresponding update rule. The new Hopfield network can store exponentially (with the dimension of the associative space) many patterns, retrieves the pattern with one update, and has exponentially small retrieval errors. We demonstrate the broad applicability of the Hopfield layers across various domains.
  arXiv  Detail & Related papers  (2020-07-16T17:52:37Z)
• LFD-ProtoNet: Prototypical Network Based on Local Fisher Discriminant Analysis for Few-shot Learning [98.64231310584614]
  The prototypical network (ProtoNet) is a few-shot learning framework that performs metric learning and classification using the distance to prototype representations of each class. We show the usefulness of the proposed method by theoretically providing an expected risk bound and empirically demonstrating its superior classification accuracy on miniImageNet and tieredImageNet.
  arXiv  Detail & Related papers  (2020-06-15T11:56:30Z)
• Text Classification with Few Examples using Controlled Generalization [58.971750512415134]
  Current practice relies on pre-trained word embeddings to map words unseen in training to similar seen ones. Our alternative begins with sparse pre-trained representations derived from unlabeled parsed corpora. We show that a feed-forward network over these vectors is especially effective in low-data scenarios.
  arXiv  Detail & Related papers  (2020-05-18T06:04:58Z)
• Fine-Grained Visual Classification with Efficient End-to-end Localization [49.9887676289364]
  We present an efficient localization module that can be fused with a classification network in an end-to-end setup. We evaluate the new model on the three benchmark datasets CUB200-2011, Stanford Cars and FGVC-Aircraft.
  arXiv  Detail & Related papers  (2020-05-11T14:07:06Z)
• Few-Shot Learning as Domain Adaptation: Algorithm and Analysis [120.75020271706978]
  Few-shot learning uses prior knowledge learned from the seen classes to recognize the unseen classes. This class-difference-caused distribution shift can be considered as a special case of domain shift. We propose a prototypical domain adaptation network with attention (DAPNA) to explicitly tackle such a domain shift problem in a meta-learning framework.
  arXiv  Detail & Related papers  (2020-02-06T01:04:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.