Light-in-the-loop: using a photonics co-processor for scalable training of neural networks

• URL: http://arxiv.org/abs/2006.01475v2
• Date: Wed, 3 Jun 2020 14:42:49 GMT
• Title: Light-in-the-loop: using a photonics co-processor for scalable training of neural networks
• Authors: Julien Launay, Iacopo Poli, Kilian M\"uller, Igor Carron, Laurent Daudet, Florent Krzakala, Sylvain Gigan
• Abstract summary: We present the first optical co-processor able to accelerate the training phase of digitally-implemented neural networks. We demonstrate its use to train a neural network for handwritten digits recognition.
• Score: 21.153688679957337
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As neural networks grow larger and more complex and data-hungry, training costs are skyrocketing. Especially when lifelong learning is necessary, such as in recommender systems or self-driving cars, this might soon become unsustainable. In this study, we present the first optical co-processor able to accelerate the training phase of digitally-implemented neural networks. We rely on direct feedback alignment as an alternative to backpropagation, and perform the error projection step optically. Leveraging the optical random projections delivered by our co-processor, we demonstrate its use to train a neural network for handwritten digits recognition.

Related papers

• Contrastive Learning in Memristor-based Neuromorphic Systems [55.11642177631929]
  Spiking neural networks have become an important family of neuron-based models that sidestep many of the key limitations facing modern-day backpropagation-trained deep networks. In this work, we design and investigate a proof-of-concept instantiation of contrastive-signal-dependent plasticity (CSDP), a neuromorphic form of forward-forward-based, backpropagation-free learning.
  arXiv  Detail & Related papers  (2024-09-17T04:48:45Z)
• Optical training of large-scale Transformers and deep neural networks with direct feedback alignment [48.90869997343841]
  We experimentally implement a versatile and scalable training algorithm, called direct feedback alignment, on a hybrid electronic-photonic platform. An optical processing unit performs large-scale random matrix multiplications, which is the central operation of this algorithm, at speeds up to 1500 TeraOps. We study the compute scaling of our hybrid optical approach, and demonstrate a potential advantage for ultra-deep and wide neural networks.
  arXiv  Detail & Related papers  (2024-09-01T12:48:47Z)
• Genetically programmable optical random neural networks [0.0]
  We demonstrate a genetically programmable yet simple optical neural network to achieve high performances with optical random projection. By genetically programming the orientation of the scattering medium which acts as a random projection kernel, our novel technique finds an optimum kernel and improves its initial test accuracies 7-22%. Our optical computing method presents a promising approach to achieve high performance in optical neural networks with a simple and scalable design.
  arXiv  Detail & Related papers  (2024-03-19T06:55:59Z)
• Design and development of opto-neural processors for simulation of neural networks trained in image detection for potential implementation in hybrid robotics [0.0]
  Living neural networks offer advantages of lower power consumption, faster processing, and biological realism. This work proposes a simulated living neural network trained indirectly by backpropagating STDP based algorithms using precision activation by optogenetics.
  arXiv  Detail & Related papers  (2024-01-17T04:42:49Z)
• Training neural networks with end-to-end optical backpropagation [1.1602089225841632]
  We show how to implement backpropagation, an algorithm for training a neural network, using optical processes. Our approach is adaptable to various analog platforms, materials, and network structures. It demonstrates the possibility of constructing neural networks entirely reliant on analog optical processes for both training and inference tasks.
  arXiv  Detail & Related papers  (2023-08-09T21:11:26Z)
• Spiking neural network for nonlinear regression [68.8204255655161]
  Spiking neural networks carry the potential for a massive reduction in memory and energy consumption. They introduce temporal and neuronal sparsity, which can be exploited by next-generation neuromorphic hardware. A framework for regression using spiking neural networks is proposed.
  arXiv  Detail & Related papers  (2022-10-06T13:04:45Z)
• Experimentally realized in situ backpropagation for deep learning in nanophotonic neural networks [0.7627023515997987]
  We design mass-manufacturable silicon photonic neural networks that cascade our custom designed "photonic mesh" accelerator. We demonstrate in situ backpropagation for the first time to solve classification tasks. Our findings suggest a new training paradigm for photonics-accelerated artificial intelligence based entirely on a physical analog of the popular backpropagation technique.
  arXiv  Detail & Related papers  (2022-05-17T17:13:50Z)
• A photonic chip-based machine learning approach for the prediction of molecular properties [11.55177943027656]
  Photonic chip technology offers an alternative platform for implementing neural network with faster data processing and lower energy usage. We demonstrate the capability of photonic neural networks in predicting the quantum mechanical properties of molecules. Our work opens the avenue for harnessing photonic technology for large-scale machine learning applications in molecular sciences.
  arXiv  Detail & Related papers  (2022-03-03T03:15:14Z)
• Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
  We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs. By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
  arXiv  Detail & Related papers  (2022-02-01T17:11:13Z)
• Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks [80.15411508088522]
  Spiking neural networks (SNNs) have shown advantages over traditional artificial neural networks (ANNs) for low latency and high computational efficiency. We propose a novel ANN-to-SNN conversion and layer-wise learning framework for rapid and efficient pattern recognition.
  arXiv  Detail & Related papers  (2020-07-02T15:38:44Z)
• Curriculum By Smoothing [52.08553521577014]
  Convolutional Neural Networks (CNNs) have shown impressive performance in computer vision tasks such as image classification, detection, and segmentation. We propose an elegant curriculum based scheme that smoothes the feature embedding of a CNN using anti-aliasing or low-pass filters. As the amount of information in the feature maps increases during training, the network is able to progressively learn better representations of the data.
  arXiv  Detail & Related papers  (2020-03-03T07:27:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.