Plasticity-Enhanced Domain-Wall MTJ Neural Networks for Energy-Efficient Online Learning

• URL: http://arxiv.org/abs/2003.02357v1
• Date: Wed, 4 Mar 2020 22:45:59 GMT
• Title: Plasticity-Enhanced Domain-Wall MTJ Neural Networks for Energy-Efficient Online Learning
• Authors: Christopher H. Bennett, T. Patrick Xiao, Can Cui, Naimul Hassan, Otitoaleke G. Akinola, Jean Anne C. Incorvia, Alvaro Velasquez, Joseph S. Friedman, and Matthew J. Marinella
• Abstract summary: We demonstrate a multi-stage learning system realized by a promising non-volatile memory device, the domain-wall magnetic tunnel junction (DW-MTJ) We demonstrate interactions between physical properties of this device and optimal implementation of neuroscience-inspired plasticity learning rules. Our energy analysis confirms the value of the approach, as the learning budget stays below 20 $mu J$ even for large tasks used typically in machine learning.
• Score: 9.481629586734497
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Machine learning implements backpropagation via abundant training samples. We demonstrate a multi-stage learning system realized by a promising non-volatile memory device, the domain-wall magnetic tunnel junction (DW-MTJ). The system consists of unsupervised (clustering) as well as supervised sub-systems, and generalizes quickly (with few samples). We demonstrate interactions between physical properties of this device and optimal implementation of neuroscience-inspired plasticity learning rules, and highlight performance on a suite of tasks. Our energy analysis confirms the value of the approach, as the learning budget stays below 20 $\mu J$ even for large tasks used typically in machine learning.

Related papers

• Training all-mechanical neural networks for task learning through in situ backpropagation [1.3812010983144802]
  We introduce the mechanical analogue of in situ backpropagation to enable highly efficient training of mechanical neural networks. With the gradient information, we showcase the successful training of MNNs for behavior learning and machine learning tasks. Our findings pave the way for mechanical machine learning hardware and autonomous self-learning material systems.
  arXiv  Detail & Related papers  (2024-04-23T19:20:41Z)
• Self-Supervised Learning with Generative Adversarial Networks for Electron Microscopy [0.0]
  We show how self-supervised pretraining facilitates efficient fine-tuning for a spectrum of downstream tasks. We demonstrate the versatility of self-supervised pretraining across various downstream tasks in the context of electron microscopy.
  arXiv  Detail & Related papers  (2024-02-28T12:25:01Z)
• Neuro-mimetic Task-free Unsupervised Online Learning with Continual Self-Organizing Maps [56.827895559823126]
  Self-organizing map (SOM) is a neural model often used in clustering and dimensionality reduction. We propose a generalization of the SOM, the continual SOM, which is capable of online unsupervised learning under a low memory budget. Our results, on benchmarks including MNIST, Kuzushiji-MNIST, and Fashion-MNIST, show almost a two times increase in accuracy.
  arXiv  Detail & Related papers  (2024-02-19T19:11:22Z)
• Effective Intrusion Detection in Heterogeneous Internet-of-Things Networks via Ensemble Knowledge Distillation-based Federated Learning [52.6706505729803]
  We introduce Federated Learning (FL) to collaboratively train a decentralized shared model of Intrusion Detection Systems (IDS) FLEKD enables a more flexible aggregation method than conventional model fusion techniques. Experiment results show that the proposed approach outperforms local training and traditional FL in terms of both speed and performance.
  arXiv  Detail & Related papers  (2024-01-22T14:16:37Z)
• Random resistive memory-based deep extreme point learning machine for unified visual processing [67.51600474104171]
  We propose a novel hardware-software co-design, random resistive memory-based deep extreme point learning machine (DEPLM) Our co-design system achieves huge energy efficiency improvements and training cost reduction when compared to conventional systems.
  arXiv  Detail & Related papers  (2023-12-14T09:46:16Z)
• MultiIoT: Benchmarking Machine Learning for the Internet of Things [70.74131118309967]
  The next generation of machine learning systems must be adept at perceiving and interacting with the physical world. sensory data from motion, thermal, geolocation, depth, wireless signals, video, and audio are increasingly used to model the states of physical environments. Existing efforts are often specialized to a single sensory modality or prediction task. This paper proposes MultiIoT, the most expansive and unified IoT benchmark to date, encompassing over 1.15 million samples from 12 modalities and 8 real-world tasks.
  arXiv  Detail & Related papers  (2023-11-10T18:13:08Z)
• Evolutionary Deep Nets for Non-Intrusive Load Monitoring [5.415995239349699]
  Non-Intrusive Load Monitoring (NILM) is an energy efficiency technique to track electricity consumption of an individual appliance in a household by one aggregated single. Deep learning approaches are implemented to operate the desegregations.
  arXiv  Detail & Related papers  (2023-03-06T22:47:40Z)
• Efficient Model-Based Multi-Agent Mean-Field Reinforcement Learning [89.31889875864599]
  We propose an efficient model-based reinforcement learning algorithm for learning in multi-agent systems. Our main theoretical contributions are the first general regret bounds for model-based reinforcement learning for MFC. We provide a practical parametrization of the core optimization problem.
  arXiv  Detail & Related papers  (2021-07-08T18:01:02Z)
• GEM: Group Enhanced Model for Learning Dynamical Control Systems [78.56159072162103]
  We build effective dynamical models that are amenable to sample-based learning. We show that learning the dynamics on a Lie algebra vector space is more effective than learning a direct state transition model. This work sheds light on a connection between learning of dynamics and Lie group properties, which opens doors for new research directions.
  arXiv  Detail & Related papers  (2021-04-07T01:08:18Z)
• Distributed Online Learning with Multiple Kernels [10.203602318836444]
  We present a privacy-preserving distributed online learning framework with multiplekernels (named DOMKL) We theoretically prove that DOMKL over T time slots can achieve an optimal sublinear regret. We verify the effectiveness of the proposed DOMKL on regression and time-series prediction tasks.
  arXiv  Detail & Related papers  (2020-11-17T20:29:00Z)
• Distributed Learning on Heterogeneous Resource-Constrained Devices [3.6187468775839373]
  We consider a distributed system, consisting of a heterogeneous set of devices, ranging from low-end to high-end. We propose the first approach that enables distributed learning in such a heterogeneous system. Applying our approach, each device employs a neural network (NN) with a topology that fits its capabilities; however, part of these NNs share the same topology, so that their parameters can be jointly learned.
  arXiv  Detail & Related papers  (2020-06-09T16:58:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.