Learning in a Multifield Coherent Ising Machine

• URL: http://arxiv.org/abs/2502.12020v1
• Date: Mon, 17 Feb 2025 16:54:54 GMT
• Title: Learning in a Multifield Coherent Ising Machine
• Authors: Daan de Bos, Marc Serra-Garcia,
• Abstract summary: We introduce a physical model for self-learning that encodes the learning rule in the Hamiltonian of the system. We numerically demonstrate that, in the presence of suitable nonlinear interactions between the long-term memory Ising machine and the short-term memory auxiliary field, the system autonomously learns from examples.
• Score: 0.0
• License:
• Abstract: Physical information processors can learn from examples if they are modified according to an abstract parameter update equation, termed a learning rule. We introduce a physical model for self-learning that encodes the learning rule in the Hamiltonian of the system. The model consists of a network of multi-modal resonators. One of the modes is driven parametrically into a bi-stable regime, forming a coherent Ising machine (CIM) -- that provides the long-term memory that stores learned responses (weights). The CIM is augmented with an additional spinor field that acts as short-term (activation) memory. We numerically demonstrate that, in the presence of suitable nonlinear interactions between the long-term memory Ising machine and the short-term memory auxiliary field, the system autonomously learns from examples.

Related papers

• Multilinear Kernel Regression and Imputation via Manifold Learning [5.482532589225551]
  MultiL-KRIM builds on the intuitive concept of spaces to tangent and incorporates collaboration among point-cloud neighbors (regressors) directly into the data-modeling term of the loss function. Two important application domains showcase the functionality of MultiL-KRIM: time-varying-graph-signal (TVGS) recovery, and reconstruction of highly accelerated dynamic-magnetic-resonance-imaging (dMRI) data.
  arXiv  Detail & Related papers  (2024-02-06T02:50:42Z)
• Sparse Modular Activation for Efficient Sequence Modeling [94.11125833685583]
  Recent models combining Linear State Space Models with self-attention mechanisms have demonstrated impressive results across a range of sequence modeling tasks. Current approaches apply attention modules statically and uniformly to all elements in the input sequences, leading to sub-optimal quality-efficiency trade-offs. We introduce Sparse Modular Activation (SMA), a general mechanism enabling neural networks to sparsely activate sub-modules for sequence elements in a differentiable manner.
  arXiv  Detail & Related papers  (2023-06-19T23:10:02Z)
• The Expressive Leaky Memory Neuron: an Efficient and Expressive Phenomenological Neuron Model Can Solve Long-Horizon Tasks [64.08042492426992]
  We introduce the Expressive Memory (ELM) neuron model, a biologically inspired model of a cortical neuron. Our ELM neuron can accurately match the aforementioned input-output relationship with under ten thousand trainable parameters. We evaluate it on various tasks with demanding temporal structures, including the Long Range Arena (LRA) datasets.
  arXiv  Detail & Related papers  (2023-06-14T13:34:13Z)
• Semiparametric Language Models Are Scalable Continual Learners [83.74414880208334]
  Semiparametric language models (LMs) have shown promise in continuously learning from new text data. We present a simple and intuitive approach called Selective Memorization (SeMem) SeMem only memorizes difficult samples that the model is likely to struggle with.
  arXiv  Detail & Related papers  (2023-03-02T17:15:02Z)
• Sparse Coding in a Dual Memory System for Lifelong Learning [13.041607703862724]
  Brain efficiently encodes information in non-overlapping sparse codes. We employ sparse coding in a multiple-memory replay mechanism. Our method maintains an additional long-term semantic memory that aggregates and consolidates information encoded in the synaptic weights of the working model.
  arXiv  Detail & Related papers  (2022-12-28T12:56:15Z)
• Dynamically-Scaled Deep Canonical Correlation Analysis [77.34726150561087]
  Canonical Correlation Analysis (CCA) is a method for feature extraction of two views by finding maximally correlated linear projections of them. We introduce a novel dynamic scaling method for training an input-dependent canonical correlation model.
  arXiv  Detail & Related papers  (2022-03-23T12:52:49Z)
• Deep Explicit Duration Switching Models for Time Series [84.33678003781908]
  We propose a flexible model that is capable of identifying both state- and time-dependent switching dynamics. State-dependent switching is enabled by a recurrent state-to-switch connection. An explicit duration count variable is used to improve the time-dependent switching behavior.
  arXiv  Detail & Related papers  (2021-10-26T17:35:21Z)
• A Framework for Machine Learning of Model Error in Dynamical Systems [7.384376731453594]
  We present a unifying framework for blending mechanistic and machine-learning approaches to identify dynamical systems from data. We cast the problem in both continuous- and discrete-time, for problems in which the model error is memoryless and in which it has significant memory. We find that hybrid methods substantially outperform solely data-driven approaches in terms of data hunger, demands for model complexity, and overall predictive performance.
  arXiv  Detail & Related papers  (2021-07-14T12:47:48Z)
• Adaptive Semiparametric Language Models [17.53604394786977]
  We present a language model that combines a large parametric neural network (i.e., a transformer) with a non-parametric episodic memory component. Experiments on word-based and character-based language modeling datasets demonstrate the efficacy of our proposed method.
  arXiv  Detail & Related papers  (2021-02-04T11:47:03Z)
• Theory of gating in recurrent neural networks [5.672132510411465]
  Recurrent neural networks (RNNs) are powerful dynamical models, widely used in machine learning (ML) and neuroscience. Here, we show that gating offers flexible control of two salient features of the collective dynamics. The gate controlling timescales leads to a novel, marginally stable state, where the network functions as a flexible integrator.
  arXiv  Detail & Related papers  (2020-07-29T13:20:58Z)
• Incremental Training of a Recurrent Neural Network Exploiting a Multi-Scale Dynamic Memory [79.42778415729475]
  We propose a novel incrementally trained recurrent architecture targeting explicitly multi-scale learning. We show how to extend the architecture of a simple RNN by separating its hidden state into different modules. We discuss a training algorithm where new modules are iteratively added to the model to learn progressively longer dependencies.
  arXiv  Detail & Related papers  (2020-06-29T08:35:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.