All-Optical Segmentation via Diffractive Neural Networks for Autonomous Driving

• URL: http://arxiv.org/abs/2602.07717v1
• Date: Sat, 07 Feb 2026 21:47:37 GMT
• Title: All-Optical Segmentation via Diffractive Neural Networks for Autonomous Driving
• Authors: Yingjie Li, Daniel Robinson, Cunxi Yu,
• Abstract summary: We propose a novel all-optical computing framework for RGB image segmentation and lane detection in autonomous driving applications.<n>Our experimental results demonstrate the effectiveness of the DONN system for image segmentation on the CityScapes dataset.<n>We conduct case studies on lane detection using a customized indoor track dataset and simulated driving scenarios in CARLA.
• Score: 10.582217646354
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Semantic segmentation and lane detection are crucial tasks in autonomous driving systems. Conventional approaches predominantly rely on deep neural networks (DNNs), which incur high energy costs due to extensive analog-to-digital conversions and large-scale image computations required for low-latency, real-time responses. Diffractive optical neural networks (DONNs) have shown promising advantages over conventional DNNs on digital or optoelectronic computing platforms in energy efficiency. By performing all-optical image processing via light diffraction at the speed of light, DONNs save computation energy costs while reducing the overhead associated with analog-to-digital conversions by all-optical encoding and computing. In this work, we propose a novel all-optical computing framework for RGB image segmentation and lane detection in autonomous driving applications. Our experimental results demonstrate the effectiveness of the DONN system for image segmentation on the CityScapes dataset. Additionally, we conduct case studies on lane detection using a customized indoor track dataset and simulated driving scenarios in CARLA, where we further evaluate the model's generalizability under diverse environmental conditions.

Related papers

• Online unsupervised Hebbian learning in deep photonic neuromorphic networks [10.099714133516608]
  Photonic neuromorphic networks (PNNs) leverage the inherent advantages of light, namely high parallelism, low latency, and exceptional energy efficiency.<n>Here, we introduce a purely photonic deep PNN architecture that enables online, unsupervised learning.<n>We experimentally demonstrate this approach on a non-trivial letter recognition task using a commercially available fiber-optic platform and achieve a 100 percent recognition rate.
  arXiv  Detail & Related papers  (2026-01-29T20:26:36Z)
• Streamlined optical training of large-scale modern deep learning architectures with direct feedback alignment [48.90869997343841]
  We experimentally implement a versatile and scalable training algorithm, called direct feedback alignment, on a hybrid electronic-photonic platform.<n>An optical processing unit performs large-scale random matrix multiplications, at speeds up to 1500 TeraOPS under 30 Watts of power.<n>We study the scaling of the training time, and demonstrate a potential advantage of our hybrid opto-electronic approach for ultra-deep and wide neural networks.
  arXiv  Detail & Related papers  (2024-09-01T12:48:47Z)
• Efficient Visual State Space Model for Image Deblurring [99.54894198086852]
  Convolutional neural networks (CNNs) and Vision Transformers (ViTs) have achieved excellent performance in image restoration.<n>We propose a simple yet effective visual state space model (EVSSM) for image deblurring.<n>The proposed EVSSM performs favorably against state-of-the-art methods on benchmark datasets and real-world images.
  arXiv  Detail & Related papers  (2024-05-23T09:13:36Z)
• Free-Space Optical Spiking Neural Network [0.0]
  We introduce the Free-space Optical deep Spiking Convolutional Neural Network (OSCNN) This novel approach draws inspiration from computational models of the human eye. Our results demonstrate promising performance with minimal latency and power consumption compared to their electronic ONN counterparts.
  arXiv  Detail & Related papers  (2023-11-08T09:41:14Z)
• Physics-aware Differentiable Discrete Codesign for Diffractive Optical Neural Networks [12.952987240366781]
  This work proposes a novel device-to-system hardware-software codesign framework, which enables efficient training of Diffractive optical neural networks (DONNs) Gumbel-Softmax is employed to enable differentiable discrete mapping from real-world device parameters into the forward function of DONNs. The results have demonstrated that our proposed framework offers significant advantages over conventional quantization-based methods.
  arXiv  Detail & Related papers  (2022-09-28T17:13:28Z)
• Optical Neural Ordinary Differential Equations [44.97261923694945]
  We propose the optical neural ordinary differential equations (ON-ODE) architecture that parameterizes the continuous dynamics of hidden layers with optical ODE solvers. The ON-ODE comprises the PNNs followed by the photonic integrator and optical feedback loop, which can be configured to represent residual neural networks (ResNet) and recurrent neural networks with effectively reduced chip area occupancy.
  arXiv  Detail & Related papers  (2022-09-26T04:04:02Z)
• Image sensing with multilayer, nonlinear optical neural networks [4.252754174399026]
  An emerging image-sensing paradigm breaks this delineation between data collection and analysis. By optically encoding images into a compressed, low-dimensional latent space suitable for efficient post-analysis, these image sensors can operate with fewer pixels and fewer photons. We demonstrate a multilayer ONN pre-processor for image sensing, using a commercial image intensifier as a parallel optoelectronic, optical-to-optical nonlinear activation function.
  arXiv  Detail & Related papers  (2022-07-27T21:00:31Z)
• All-optical graph representation learning using integrated diffractive photonic computing units [51.15389025760809]
  Photonic neural networks perform brain-inspired computations using photons instead of electrons. We propose an all-optical graph representation learning architecture, termed diffractive graph neural network (DGNN) We demonstrate the use of DGNN extracted features for node and graph-level classification tasks with benchmark databases and achieve superior performance.
  arXiv  Detail & Related papers  (2022-04-23T02:29:48Z)
• Hybrid SNN-ANN: Energy-Efficient Classification and Object Detection for Event-Based Vision [64.71260357476602]
  Event-based vision sensors encode local pixel-wise brightness changes in streams of events rather than image frames. Recent progress in object recognition from event-based sensors has come from conversions of deep neural networks. We propose a hybrid architecture for end-to-end training of deep neural networks for event-based pattern recognition and object detection.
  arXiv  Detail & Related papers  (2021-12-06T23:45:58Z)
• Learning Frequency-aware Dynamic Network for Efficient Super-Resolution [56.98668484450857]
  This paper explores a novel frequency-aware dynamic network for dividing the input into multiple parts according to its coefficients in the discrete cosine transform (DCT) domain. In practice, the high-frequency part will be processed using expensive operations and the lower-frequency part is assigned with cheap operations to relieve the computation burden. Experiments conducted on benchmark SISR models and datasets show that the frequency-aware dynamic network can be employed for various SISR neural architectures.
  arXiv  Detail & Related papers  (2021-03-15T12:54:26Z)
• Scale-, shift- and rotation-invariant diffractive optical networks [0.0]
  Diffractive Deep Neural Networks (D2NNs) harness light-matter interaction over a series of trainable surfaces to compute a desired statistical inference task. Here, we demonstrate a new training strategy for diffractive networks that introduces input object translation, rotation and/or scaling during the training phase. This training strategy successfully guides the evolution of the diffractive optical network design towards a solution that is scale-, shift- and rotation-invariant.
  arXiv  Detail & Related papers  (2020-10-24T02:18:39Z)
• Temporal Pulses Driven Spiking Neural Network for Fast Object Recognition in Autonomous Driving [65.36115045035903]
  We propose an approach to address the object recognition problem directly with raw temporal pulses utilizing the spiking neural network (SNN) Being evaluated on various datasets, our proposed method has shown comparable performance as the state-of-the-art methods, while achieving remarkable time efficiency.
  arXiv  Detail & Related papers  (2020-01-24T22:58:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.