Opto-Electronic Convolutional Neural Network Design Via Direct Kernel Optimization

• URL: http://arxiv.org/abs/2511.02065v1
• Date: Mon, 03 Nov 2025 21:01:41 GMT
• Title: Opto-Electronic Convolutional Neural Network Design Via Direct Kernel Optimization
• Authors: Ali Almuallem, Harshana Weligampola, Abhiram Gnanasambandam, Wei Xu, Dilshan Godaliyadda, Hamid R. Sheikh, Stanley H. Chan, Qi Guo,
• Abstract summary: Opto-electronic neural networks integrate optical front-ends with electronic back-ends to enable fast and energy-efficient vision.<n>We introduce a two-stage strategy for designing opto-electronic convolutional neural networks (CNNs)<n>First, train a standard electronic CNN, then realize the optical front-end implemented as a metasurface array through direct kernel optimization of its first convolutional layer.<n>This approach reduces computational and memory demands by hundreds of times and improves training stability compared to end-to-end optimization.
• Score: 24.845064567059556
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Opto-electronic neural networks integrate optical front-ends with electronic back-ends to enable fast and energy-efficient vision. However, conventional end-to-end optimization of both the optical and electronic modules is limited by costly simulations and large parameter spaces. We introduce a two-stage strategy for designing opto-electronic convolutional neural networks (CNNs): first, train a standard electronic CNN, then realize the optical front-end implemented as a metasurface array through direct kernel optimization of its first convolutional layer. This approach reduces computational and memory demands by hundreds of times and improves training stability compared to end-to-end optimization. On monocular depth estimation, the proposed two-stage design achieves twice the accuracy of end-to-end training under the same training time and resource constraints.

Related papers

• First Demonstration of Second-order Training of Deep Neural Networks with In-memory Analog Matrix Computing [7.51466944063829]
  We present the first demonstration of a second-order powered by in-memory analog matrix computing (AMC)<n>We validate the inversion by training a two-layer convolutional neural network (CNN) for handwritten letter classification.<n>Our system delivers a 5.88x improvement in throughput and a 6.9x gain in energy efficiency compared to state-of-the-art digital processors.
  arXiv  Detail & Related papers  (2025-12-05T00:52:46Z)
• Streamlined optical training of large-scale modern deep learning architectures with direct feedback alignment [48.90869997343841]
  We experimentally implement a versatile and scalable training algorithm, called direct feedback alignment, on a hybrid electronic-photonic platform.<n>An optical processing unit performs large-scale random matrix multiplications, at speeds up to 1500 TeraOPS under 30 Watts of power.<n>We study the scaling of the training time, and demonstrate a potential advantage of our hybrid opto-electronic approach for ultra-deep and wide neural networks.
  arXiv  Detail & Related papers  (2024-09-01T12:48:47Z)
• Auto-Train-Once: Controller Network Guided Automatic Network Pruning from Scratch [72.26822499434446]
  Auto-Train-Once (ATO) is an innovative network pruning algorithm designed to automatically reduce the computational and storage costs of DNNs. We provide a comprehensive convergence analysis as well as extensive experiments, and the results show that our approach achieves state-of-the-art performance across various model architectures.
  arXiv  Detail & Related papers  (2024-03-21T02:33:37Z)
• Real-Time Machine-Learning-Based Optimization Using Input Convex Long Short-Term Memory Network [18.84965086425835]
  We present a novel input memory-based neural network-based optimization for energy and chemical systems. We demonstrate the superior performance of ICLSTM-based optimization in terms of runtime. Specifically, in a real-time optimization problem of a real-world energy system at LHT Holdings in Singapore, IC-LSTM-based optimization achieved at least 4-fold compared to conventional LSTM-based optimization.
  arXiv  Detail & Related papers  (2023-11-13T09:41:32Z)
• Towards Hyperparameter-Agnostic DNN Training via Dynamical System Insights [4.513581513983453]
  We present a first-order optimization method specialized for deep neural networks (DNNs), ECCO-DNN. This method models the optimization variable trajectory as a dynamical system and develops a discretization algorithm that adaptively selects step sizes based on the trajectory's shape.
  arXiv  Detail & Related papers  (2023-10-21T03:45:13Z)
• Speed Limits for Deep Learning [67.69149326107103]
  Recent advancement in thermodynamics allows bounding the speed at which one can go from the initial weight distribution to the final distribution of the fully trained network. We provide analytical expressions for these speed limits for linear and linearizable neural networks. Remarkably, given some plausible scaling assumptions on the NTK spectra and spectral decomposition of the labels -- learning is optimal in a scaling sense.
  arXiv  Detail & Related papers  (2023-07-27T06:59:46Z)
• Deep learning based Meta-modeling for Multi-objective Technology Optimization of Electrical Machines [0.0]
  We present the application of a variational auto-encoder to optimize two different machine technologies simultaneously. After training, we employ a deep neural network and a decoder as meta-models to predict global key performance indicators.
  arXiv  Detail & Related papers  (2023-06-15T12:33:39Z)
• Graph Neural Network Autoencoders for Efficient Quantum Circuit Optimisation [69.43216268165402]
  We present for the first time how to use graph neural network (GNN) autoencoders for the optimisation of quantum circuits. We construct directed acyclic graphs from the quantum circuits, encode the graphs and use the encodings to represent RL states. Our method is the first realistic first step towards very large scale RL quantum circuit optimisation.
  arXiv  Detail & Related papers  (2023-03-06T16:51:30Z)
• Efficient On-Chip Learning for Optical Neural Networks Through Power-Aware Sparse Zeroth-Order Optimization [12.052076188811052]
  Optical neural networks (ONNs) have demonstrated record-breaking potential in neuromorphic computing. We propose a novel on-chip learning framework to release the full potential of ONNs for power-efficient in situ training.
  arXiv  Detail & Related papers  (2020-12-21T07:00:39Z)
• Adaptive pruning-based optimization of parameterized quantum circuits [62.997667081978825]
  Variisy hybrid quantum-classical algorithms are powerful tools to maximize the use of Noisy Intermediate Scale Quantum devices. We propose a strategy for such ansatze used in variational quantum algorithms, which we call "Efficient Circuit Training" (PECT) Instead of optimizing all of the ansatz parameters at once, PECT launches a sequence of variational algorithms.
  arXiv  Detail & Related papers  (2020-10-01T18:14:11Z)
• Optimal Gradient Quantization Condition for Communication-Efficient Distributed Training [99.42912552638168]
  Communication of gradients is costly for training deep neural networks with multiple devices in computer vision applications. In this work, we deduce the optimal condition of both the binary and multi-level gradient quantization for textbfANY gradient distribution. Based on the optimal condition, we develop two novel quantization schemes: biased BinGrad and unbiased ORQ for binary and multi-level gradient quantization respectively.
  arXiv  Detail & Related papers  (2020-02-25T18:28:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.