Spectrally-Encoded Single-Pixel Machine Vision Using Diffractive Networks

• URL: http://arxiv.org/abs/2005.11387v2
• Date: Fri, 26 Mar 2021 04:48:42 GMT
• Title: Spectrally-Encoded Single-Pixel Machine Vision Using Diffractive Networks
• Authors: Jingxi Li, Deniz Mengu, Nezih T. Yardimci, Yi Luo, Xurong Li, Muhammed Veli, Yair Rivenson, Mona Jarrahi, Aydogan Ozcan
• Abstract summary: 3D engineering of matter has opened up new avenues for designing systems that can perform various computational tasks through light-matter interaction. Here, we demonstrate the design of optical networks in the form of multiple diffractive layers that are trained using deep learning to transform and encode the spatial information of objects into the power spectrum of the diffracted light. We experimentally validated this machine vision framework at terahertz spectrum to optically classify the images of handwritten digits by detecting the spectral power of the diffracted light at ten distinct wavelengths.
• Score: 6.610893384480686
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: 3D engineering of matter has opened up new avenues for designing systems that can perform various computational tasks through light-matter interaction. Here, we demonstrate the design of optical networks in the form of multiple diffractive layers that are trained using deep learning to transform and encode the spatial information of objects into the power spectrum of the diffracted light, which are used to perform optical classification of objects with a single-pixel spectroscopic detector. Using a time-domain spectroscopy setup with a plasmonic nanoantenna-based detector, we experimentally validated this machine vision framework at terahertz spectrum to optically classify the images of handwritten digits by detecting the spectral power of the diffracted light at ten distinct wavelengths, each representing one class/digit. We also report the coupling of this spectral encoding achieved through a diffractive optical network with a shallow electronic neural network, separately trained to reconstruct the images of handwritten digits based on solely the spectral information encoded in these ten distinct wavelengths within the diffracted light. These reconstructed images demonstrate task-specific image decompression and can also be cycled back as new inputs to the same diffractive network to improve its optical object classification. This unique machine vision framework merges the power of deep learning with the spatial and spectral processing capabilities of diffractive networks, and can also be extended to other spectral-domain measurement systems to enable new 3D imaging and sensing modalities integrated with spectrally encoded classification tasks performed through diffractive optical networks.

Related papers

• TOP-ReID: Multi-spectral Object Re-Identification with Token Permutation [64.65950381870742]
  We propose a cyclic token permutation framework for multi-spectral object ReID, dubbled TOP-ReID. We also propose a Token Permutation Module (TPM) for cyclic multi-spectral feature aggregation. Our proposed framework can generate more discriminative multi-spectral features for robust object ReID.
  arXiv  Detail & Related papers  (2023-12-15T08:54:15Z)
• SpectralGPT: Spectral Remote Sensing Foundation Model [60.023956954916414]
  A universal RS foundation model, named SpectralGPT, is purpose-built to handle spectral RS images using a novel 3D generative pretrained transformer (GPT) Compared to existing foundation models, SpectralGPT accommodates input images with varying sizes, resolutions, time series, and regions in a progressive training fashion, enabling full utilization of extensive RS big data. Our evaluation highlights significant performance improvements with pretrained SpectralGPT models, signifying substantial potential in advancing spectral RS big data applications within the field of geoscience.
  arXiv  Detail & Related papers  (2023-11-13T07:09:30Z)
• Snapshot Multispectral Imaging Using a Diffractive Optical Network [2.8880000014100506]
  We present a diffractive optical network-based multispectral imaging system trained using deep learning. This diffractive multispectral imager performs spatially-coherent imaging over a large spectrum. We experimentally demonstrate a diffractive multispectral imager based on a 3D-printed diffractive network.
  arXiv  Detail & Related papers  (2022-12-10T05:54:24Z)
• Time-lapse image classification using a diffractive neural network [0.0]
  We show for the first time a time-lapse image classification scheme using a diffractive network. We show a blind testing accuracy of 62.03% on the optical classification of objects from the CIFAR-10 dataset. This constitutes the highest inference accuracy achieved so far using a single diffractive network.
  arXiv  Detail & Related papers  (2022-08-23T08:16:30Z)
• All-optical image classification through unknown random diffusers using a single-pixel diffractive network [13.7472825798265]
  classification of an object behind a random and unknown scattering medium sets a challenging task for computational imaging and machine vision fields. Recent deep learning-based approaches demonstrated the classification of objects using diffuser-distorted patterns collected by an image sensor. Here, we present an all-optical processor to directly classify unknown objects through unknown, random phase diffusers using broadband illumination detected with a single pixel.
  arXiv  Detail & Related papers  (2022-08-08T08:26:08Z)
• Deep Coding Patterns Design for Compressive Near-Infrared Spectral Classification [80.93625278357229]
  spectral classification can be performed directly in the compressive domain, considering the amount of spectral information embedded in the measurements. This work proposes an end-to-end approach to jointly design the coding patterns used in CSI and the network parameters to perform spectral classification directly from the embedded near-infrared compressive measurements.
  arXiv  Detail & Related papers  (2022-05-27T15:55:53Z)
• All-optical graph representation learning using integrated diffractive photonic computing units [51.15389025760809]
  Photonic neural networks perform brain-inspired computations using photons instead of electrons. We propose an all-optical graph representation learning architecture, termed diffractive graph neural network (DGNN) We demonstrate the use of DGNN extracted features for node and graph-level classification tasks with benchmark databases and achieve superior performance.
  arXiv  Detail & Related papers  (2022-04-23T02:29:48Z)
• Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction [127.20208645280438]
  Hyperspectral image (HSI) reconstruction aims to recover the 3D spatial-spectral signal from a 2D measurement. Modeling the inter-spectra interactions is beneficial for HSI reconstruction. Mask-guided Spectral-wise Transformer (MST) proposes a novel framework for HSI reconstruction.
  arXiv  Detail & Related papers  (2021-11-15T16:59:48Z)
• Neural network-based on-chip spectroscopy using a scalable plasmonic encoder [0.4397520291340694]
  Conventional spectrometers are limited by trade-offs set by size, cost, signal-to-noise ratio (SNR), and spectral resolution. Here, we demonstrate a deep learning-based spectral reconstruction framework using a compact and low-cost on-chip sensing scheme.
  arXiv  Detail & Related papers  (2020-12-01T22:50:06Z)
• Spectral Analysis Network for Deep Representation Learning and Image Clustering [53.415803942270685]
  This paper proposes a new network structure for unsupervised deep representation learning based on spectral analysis. It can identify the local similarities among images in patch level and thus more robust against occlusion. It can learn more clustering-friendly representations and is capable to reveal the deep correlations among data samples.
  arXiv  Detail & Related papers  (2020-09-11T05:07:15Z)
• Terahertz Pulse Shaping Using Diffractive Surfaces [6.895625925414448]
  We present a diffractive network, which is used to shape an arbitrary broadband pulse into a desired optical waveform. Results constitute the first demonstration of direct pulse shaping in terahertz spectrum. This learning-based diffractive pulse engineering framework can find broad applications in e.g., communications, ultra-fast imaging and spectroscopy.
  arXiv  Detail & Related papers  (2020-06-30T08:27:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.