Hybrid training of optical neural networks
- URL: http://arxiv.org/abs/2203.11207v1
- Date: Sun, 20 Mar 2022 21:16:42 GMT
- Title: Hybrid training of optical neural networks
- Authors: James Spall, Xianxin Guo, and A. I. Lvovsky
- Abstract summary: Optical neural networks are emerging as a promising type of machine learning hardware.
These networks are mainly developed to perform optical inference after in silico training on digital simulators.
We show that hybrid training of optical neural networks can be applied to a wide variety of optical neural networks.
- Score: 1.0323063834827415
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Optical neural networks are emerging as a promising type of machine learning
hardware capable of energy-efficient, parallel computation. Today's optical
neural networks are mainly developed to perform optical inference after in
silico training on digital simulators. However, various physical imperfections
that cannot be accurately modelled may lead to the notorious reality gap
between the digital simulator and the physical system. To address this
challenge, we demonstrate hybrid training of optical neural networks where the
weight matrix is trained with neuron activation functions computed optically
via forward propagation through the network. We examine the efficacy of hybrid
training with three different networks: an optical linear classifier, a hybrid
opto-electronic network, and a complex-valued optical network. We perform a
comparative study to in silico training, and our results show that hybrid
training is robust against different kinds of static noise. Our
platform-agnostic hybrid training scheme can be applied to a wide variety of
optical neural networks, and this work paves the way towards advanced
all-optical training in machine intelligence.
Related papers
- Genetically programmable optical random neural networks [0.0]
We demonstrate a genetically programmable yet simple optical neural network to achieve high performances with optical random projection.
By genetically programming the orientation of the scattering medium which acts as a random projection kernel, our novel technique finds an optimum kernel and improves its initial test accuracies 7-22%.
Our optical computing method presents a promising approach to achieve high performance in optical neural networks with a simple and scalable design.
arXiv Detail & Related papers (2024-03-19T06:55:59Z) - Design and development of opto-neural processors for simulation of
neural networks trained in image detection for potential implementation in
hybrid robotics [0.0]
Living neural networks offer advantages of lower power consumption, faster processing, and biological realism.
This work proposes a simulated living neural network trained indirectly by backpropagating STDP based algorithms using precision activation by optogenetics.
arXiv Detail & Related papers (2024-01-17T04:42:49Z) - Training neural networks with end-to-end optical backpropagation [1.1602089225841632]
We show how to implement backpropagation, an algorithm for training a neural network, using optical processes.
Our approach is adaptable to various analog platforms, materials, and network structures.
It demonstrates the possibility of constructing neural networks entirely reliant on analog optical processes for both training and inference tasks.
arXiv Detail & Related papers (2023-08-09T21:11:26Z) - Contrastive-Signal-Dependent Plasticity: Forward-Forward Learning of
Spiking Neural Systems [73.18020682258606]
We develop a neuro-mimetic architecture, composed of spiking neuronal units, where individual layers of neurons operate in parallel.
We propose an event-based generalization of forward-forward learning, which we call contrastive-signal-dependent plasticity (CSDP)
Our experimental results on several pattern datasets demonstrate that the CSDP process works well for training a dynamic recurrent spiking network capable of both classification and reconstruction.
arXiv Detail & Related papers (2023-03-30T02:40:28Z) - Unsupervised Domain Transfer with Conditional Invertible Neural Networks [83.90291882730925]
We propose a domain transfer approach based on conditional invertible neural networks (cINNs)
Our method inherently guarantees cycle consistency through its invertible architecture, and network training can efficiently be conducted with maximum likelihood.
Our method enables the generation of realistic spectral data and outperforms the state of the art on two downstream classification tasks.
arXiv Detail & Related papers (2023-03-17T18:00:27Z) - Experimentally realized in situ backpropagation for deep learning in
nanophotonic neural networks [0.7627023515997987]
We design mass-manufacturable silicon photonic neural networks that cascade our custom designed "photonic mesh" accelerator.
We demonstrate in situ backpropagation for the first time to solve classification tasks.
Our findings suggest a new training paradigm for photonics-accelerated artificial intelligence based entirely on a physical analog of the popular backpropagation technique.
arXiv Detail & Related papers (2022-05-17T17:13:50Z) - Photonics for artificial intelligence and neuromorphic computing [52.77024349608834]
Photonic integrated circuits have enabled ultrafast artificial neural networks.
Photonic neuromorphic systems offer sub-nanosecond latencies.
These systems could address the growing demand for machine learning and artificial intelligence.
arXiv Detail & Related papers (2020-10-30T21:41:44Z) - Rapid characterisation of linear-optical networks via PhaseLift [51.03305009278831]
Integrated photonics offers great phase-stability and can rely on the large scale manufacturability provided by the semiconductor industry.
New devices, based on such optical circuits, hold the promise of faster and energy-efficient computations in machine learning applications.
We present a novel technique to reconstruct the transfer matrix of linear optical networks.
arXiv Detail & Related papers (2020-10-01T16:04:22Z) - Reservoir Memory Machines as Neural Computers [70.5993855765376]
Differentiable neural computers extend artificial neural networks with an explicit memory without interference.
We achieve some of the computational capabilities of differentiable neural computers with a model that can be trained very efficiently.
arXiv Detail & Related papers (2020-09-14T12:01:30Z) - Training End-to-End Analog Neural Networks with Equilibrium Propagation [64.0476282000118]
We introduce a principled method to train end-to-end analog neural networks by gradient descent.
We show mathematically that a class of analog neural networks (called nonlinear resistive networks) are energy-based models.
Our work can guide the development of a new generation of ultra-fast, compact and low-power neural networks supporting on-chip learning.
arXiv Detail & Related papers (2020-06-02T23:38:35Z) - Light-in-the-loop: using a photonics co-processor for scalable training
of neural networks [21.153688679957337]
We present the first optical co-processor able to accelerate the training phase of digitally-implemented neural networks.
We demonstrate its use to train a neural network for handwritten digits recognition.
arXiv Detail & Related papers (2020-06-02T09:19:45Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.