Machine Learning Without a Processor: Emergent Learning in a Nonlinear Electronic Metamaterial

• URL: http://arxiv.org/abs/2311.00537v2
• Date: Fri, 5 Apr 2024 22:50:25 GMT
• Title: Machine Learning Without a Processor: Emergent Learning in a Nonlinear Electronic Metamaterial
• Authors: Sam Dillavou, Benjamin D Beyer, Menachem Stern, Andrea J Liu, Marc Z Miskin, Douglas J Durian,
• Abstract summary: We introduce a nonlinear learning metamaterial -- an analog electronic network made of self-adjusting nonlinear resistive elements based on transistors. We demonstrate that the system learns tasks unachievable in linear systems, including XOR and nonlinear regression, without a computer. This suggests enormous potential for fast, low-power computing in edge systems like sensors, robotic controllers, and medical devices.
• Score: 0.6597195879147557
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Standard deep learning algorithms require differentiating large nonlinear networks, a process that is slow and power-hungry. Electronic learning metamaterials offer potentially fast, efficient, and fault-tolerant hardware for analog machine learning, but existing implementations are linear, severely limiting their capabilities. These systems differ significantly from artificial neural networks as well as the brain, so the feasibility and utility of incorporating nonlinear elements have not been explored. Here we introduce a nonlinear learning metamaterial -- an analog electronic network made of self-adjusting nonlinear resistive elements based on transistors. We demonstrate that the system learns tasks unachievable in linear systems, including XOR and nonlinear regression, without a computer. We find our nonlinear learning metamaterial reduces modes of training error in order (mean, slope, curvature), similar to spectral bias in artificial neural networks. The circuitry is robust to damage, retrainable in seconds, and performs learned tasks in microseconds while dissipating only picojoules of energy across each transistor. This suggests enormous potential for fast, low-power computing in edge systems like sensors, robotic controllers, and medical devices, as well as manufacturability at scale for performing and studying emergent learning.

Related papers

• Coding schemes in neural networks learning classification tasks [52.22978725954347]
  We investigate fully-connected, wide neural networks learning classification tasks. We show that the networks acquire strong, data-dependent features. Surprisingly, the nature of the internal representations depends crucially on the neuronal nonlinearity.
  arXiv  Detail & Related papers  (2024-06-24T14:50:05Z)
• Machine Learning with Chaotic Strange Attractors [0.0]
  We present an analog computing method that harnesses chaotic nonlinear attractors to perform machine learning tasks with low power consumption. Inspired by neuromorphic computing, our model is a programmable, versatile, and generalized platform for machine learning tasks. When deployed as a simple analog device, it only requires milliwatt-scale power levels while being on par with current machine learning techniques.
  arXiv  Detail & Related papers  (2023-09-23T12:54:38Z)
• Neuromorphic analog circuits for robust on-chip always-on learning in spiking neural networks [1.9809266426888898]
  Mixed-signal neuromorphic systems represent a promising solution for solving extreme-edge computing tasks. Their spiking neural network circuits are optimized for processing sensory data on-line in continuous-time. We design on-chip learning circuits with short-term analog dynamics and long-term tristate discretization mechanisms.
  arXiv  Detail & Related papers  (2023-07-12T11:14:25Z)
• What Can We Learn from Unlearnable Datasets? [107.12337511216228]
  Unlearnable datasets have the potential to protect data privacy by preventing deep neural networks from generalizing. It is widely believed that neural networks trained on unlearnable datasets only learn shortcuts, simpler rules that are not useful for generalization. In contrast, we find that networks actually can learn useful features that can be reweighed for high test performance, suggesting that image protection is not assured.
  arXiv  Detail & Related papers  (2023-05-30T17:41:35Z)
• The least-control principle for learning at equilibrium [65.2998274413952]
  We present a new principle for learning equilibrium recurrent neural networks, deep equilibrium models, or meta-learning. Our results shed light on how the brain might learn and offer new ways of approaching a broad class of machine learning problems.
  arXiv  Detail & Related papers  (2022-07-04T11:27:08Z)
• A Robust Learning Rule for Soft-Bounded Memristive Synapses Competitive with Supervised Learning in Standard Spiking Neural Networks [0.0]
  A view in theoretical neuroscience sees the brain as a function-computing device. Being able to approximate functions is a fundamental axiom to build upon for future brain research. In this work we apply a novel supervised learning algorithm - based on controlling niobium-doped strontium titanate memristive synapses - to learning non-trivial multidimensional functions.
  arXiv  Detail & Related papers  (2022-04-12T10:21:22Z)
• Gradient Descent in Materio [3.756477173839499]
  We show an efficient and accurate homodyne gradient extraction method for performing gradient descent on the loss function directly in the material system. This shows that gradient descent can in principle be fully implemented in materio using simple electronics.
  arXiv  Detail & Related papers  (2021-05-15T12:18:31Z)
• Linear embedding of nonlinear dynamical systems and prospects for efficient quantum algorithms [74.17312533172291]
  We describe a method for mapping any finite nonlinear dynamical system to an infinite linear dynamical system (embedding) We then explore an approach for approximating the resulting infinite linear system with finite linear systems (truncation)
  arXiv  Detail & Related papers  (2020-12-12T00:01:10Z)
• One-step regression and classification with crosspoint resistive memory arrays [62.997667081978825]
  High speed, low energy computing machines are in demand to enable real-time artificial intelligence at the edge. One-step learning is supported by simulations of the prediction of the cost of a house in Boston and the training of a 2-layer neural network for MNIST digit recognition. Results are all obtained in one computational step, thanks to the physical, parallel, and analog computing within the crosspoint array.
  arXiv  Detail & Related papers  (2020-05-05T08:00:07Z)
• Boolean learning under noise-perturbations in hardware neural networks [0.0]
  We find that noise strongly modifies the system's path during convergence, and surprisingly fully decorrelates the final readout weight matrices. This highlights the importance of understanding architecture, noise and learning algorithm as interacting players.
  arXiv  Detail & Related papers  (2020-03-27T10:36:03Z)
• Deep Learning for Ultra-Reliable and Low-Latency Communications in 6G Networks [84.2155885234293]
  We first summarize how to apply data-driven supervised deep learning and deep reinforcement learning in URLLC. To address these open problems, we develop a multi-level architecture that enables device intelligence, edge intelligence, and cloud intelligence for URLLC.
  arXiv  Detail & Related papers  (2020-02-22T14:38:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.