Photonic Accelerators for Image Segmentation in Autonomous Driving and Defect Detection

• URL: http://arxiv.org/abs/2309.16783v2
• Date: Tue, 3 Oct 2023 16:34:13 GMT
• Title: Photonic Accelerators for Image Segmentation in Autonomous Driving and Defect Detection
• Authors: Lakshmi Nair, David Widemann, Brad Turcott, Nick Moore, Alexandra Wleklinski, Darius Bunandar, Ioannis Papavasileiou, Shihu Wang, Eric Logan
• Abstract summary: Photonic computing promises faster and more energy-efficient deep neural network (DNN) inference than traditional digital hardware. We show that certain segmentation models exhibit negligible loss in accuracy (compared to digital float32 models) when executed on photonic accelerators. We discuss the challenges and potential optimizations that can help improve the application of photonic accelerators to such computer vision tasks.
• Score: 34.864059478265055
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Photonic computing promises faster and more energy-efficient deep neural network (DNN) inference than traditional digital hardware. Advances in photonic computing can have profound impacts on applications such as autonomous driving and defect detection that depend on fast, accurate and energy efficient execution of image segmentation models. In this paper, we investigate image segmentation on photonic accelerators to explore: a) the types of image segmentation DNN architectures that are best suited for photonic accelerators, and b) the throughput and energy efficiency of executing the different image segmentation models on photonic accelerators, along with the trade-offs involved therein. Specifically, we demonstrate that certain segmentation models exhibit negligible loss in accuracy (compared to digital float32 models) when executed on photonic accelerators, and explore the empirical reasoning for their robustness. We also discuss techniques for recovering accuracy in the case of models that do not perform well. Further, we compare throughput (inferences-per-second) and energy consumption estimates for different image segmentation workloads on photonic accelerators. We discuss the challenges and potential optimizations that can help improve the application of photonic accelerators to such computer vision tasks.

Related papers

• Efficient Visual State Space Model for Image Deblurring [83.57239834238035]
  Convolutional neural networks (CNNs) and Vision Transformers (ViTs) have achieved excellent performance in image restoration. We propose a simple yet effective visual state space model (EVSSM) for image deblurring.
  arXiv  Detail & Related papers  (2024-05-23T09:13:36Z)
• EEGEncoder: Advancing BCI with Transformer-Based Motor Imagery Classification [11.687193535939798]
  Brain-computer interfaces (BCIs) harness electroencephalographic signals for direct neural control of devices. Traditional machine learning methods for EEG-based motor imagery (MI) classification encounter challenges such as manual feature extraction and susceptibility to noise. This paper introduces EEGEncoder, a deep learning framework that employs modified transformers and TCNs to surmount these limitations.
  arXiv  Detail & Related papers  (2024-04-23T09:51:24Z)
• TeMPO: Efficient Time-Multiplexed Dynamic Photonic Tensor Core for Edge AI with Compact Slow-Light Electro-Optic Modulator [44.74560543672329]
  We present a time-multiplexed dynamic photonic tensor accelerator, dubbed TeMPO, with cross-layer device/circuit/architecture customization. We achieve a 368.6 TOPS peak performance, 22.3 TOPS/W energy efficiency, and 1.2 TOPS/mm$2$ compute density. This work signifies the power of cross-layer co-design and domain-specific customization, paving the way for future electronic-photonic accelerators.
  arXiv  Detail & Related papers  (2024-02-12T03:40:32Z)
• DeltaNN: Assessing the Impact of Computational Environment Parameters on the Performance of Image Recognition Models [2.379078565066793]
  Failure in real-time image recognition tasks can occur due to sub-optimal mapping on hardware accelerators. We present a differential testing framework, DeltaNN, that allows us to assess the impact of different computational environment parameters on the performance of image recognition models.
  arXiv  Detail & Related papers  (2023-06-05T23:07:01Z)
• GDIP: Gated Differentiable Image Processing for Object-Detection in Adverse Conditions [15.327704761260131]
  We present a Gated Differentiable Image Processing (GDIP) block, a domain-agnostic network architecture. Our proposed GDIP block learns to enhance images directly through the downstream object detection loss. We demonstrate significant improvement in detection performance over several state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-09-29T16:43:13Z)
• All-optical graph representation learning using integrated diffractive photonic computing units [51.15389025760809]
  Photonic neural networks perform brain-inspired computations using photons instead of electrons. We propose an all-optical graph representation learning architecture, termed diffractive graph neural network (DGNN) We demonstrate the use of DGNN extracted features for node and graph-level classification tasks with benchmark databases and achieve superior performance.
  arXiv  Detail & Related papers  (2022-04-23T02:29:48Z)
• Illumination and Temperature-Aware Multispectral Networks for Edge-Computing-Enabled Pedestrian Detection [10.454696553567809]
  This study proposes a lightweight Illumination and Temperature-aware Multispectral Network (IT-MN) for accurate and efficient pedestrian detection. The proposed algorithm is evaluated by comparing with the selected state-of-the-art algorithms using a public dataset collected by in-vehicle cameras. The results show that the proposed algorithm achieves a low miss rate and inference time at 14.19% and 0.03 seconds per image pair on GPU.
  arXiv  Detail & Related papers  (2021-12-09T17:27:23Z)
• Learning Deformable Image Registration from Optimization: Perspective, Modules, Bilevel Training and Beyond [62.730497582218284]
  We develop a new deep learning based framework to optimize a diffeomorphic model via multi-scale propagation. We conduct two groups of image registration experiments on 3D volume datasets including image-to-atlas registration on brain MRI data and image-to-image registration on liver CT data.
  arXiv  Detail & Related papers  (2020-04-30T03:23:45Z)
• Thermal phase shifters for femtosecond laser written photonic integrated circuits [58.720142291102135]
  Photonic integrated circuits (PICs) are acknowledged as an effective solution to fulfill the demanding requirements of many practical applications in both classical and quantum optics. Phase shifters integrated in the photonic circuit offer the possibility to dynamically reconfigure its properties in order to fine tune its operation or to produce adaptive circuits. We show how thermal shifters to reconfigure photonic circuits can be solved by a careful design of the thermal shifters and by choosing the most appropriate way to drive them.
  arXiv  Detail & Related papers  (2020-04-23T20:09:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.