Ultrafast single-channel machine vision based on neuro-inspired photonic computing

• URL: http://arxiv.org/abs/2302.07875v1
• Date: Wed, 15 Feb 2023 10:08:04 GMT
• Title: Ultrafast single-channel machine vision based on neuro-inspired photonic computing
• Authors: Tomoya Yamaguchi, Kohei Arai, Tomoaki Niiyama, Atsushi Uchida, and Satoshi Sunada
• Abstract summary: Neuro-inspired photonic computing is a promising approach to speed-up machine vision processing with ultralow latency. Here, we propose an image-sensor-free machine vision framework, which optically processes real-world visual information with only a single input channel. We experimentally demonstrate that the proposed approach is capable of high-speed image recognition and anomaly detection, and furthermore, it can be used for high-speed imaging.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: High-speed machine vision is increasing its importance in both scientific and technological applications. Neuro-inspired photonic computing is a promising approach to speed-up machine vision processing with ultralow latency. However, the processing rate is fundamentally limited by the low frame rate of image sensors, typically operating at tens of hertz. Here, we propose an image-sensor-free machine vision framework, which optically processes real-world visual information with only a single input channel, based on a random temporal encoding technique. This approach allows for compressive acquisitions of visual information with a single channel at gigahertz rates, outperforming conventional approaches, and enables its direct photonic processing using a photonic reservoir computer in a time domain. We experimentally demonstrate that the proposed approach is capable of high-speed image recognition and anomaly detection, and furthermore, it can be used for high-speed imaging. The proposed approach is multipurpose and can be extended for a wide range of applications, including tracking, controlling, and capturing sub-nanosecond phenomena.

Related papers

• Optical training of large-scale Transformers and deep neural networks with direct feedback alignment [48.90869997343841]
  We experimentally implement a versatile and scalable training algorithm, called direct feedback alignment, on a hybrid electronic-photonic platform. An optical processing unit performs large-scale random matrix multiplications, which is the central operation of this algorithm, at speeds up to 1500 TeraOps. We study the compute scaling of our hybrid optical approach, and demonstrate a potential advantage for ultra-deep and wide neural networks.
  arXiv  Detail & Related papers  (2024-09-01T12:48:47Z)
• Latent Space Imaging [15.435034286180295]
  We propose to follow a similar approach for the development of artificial vision systems. Latent Space Imaging is a new paradigm that, through a combination of optics and software, directly encodes the image information into the semantically rich latent space of a generative model. We demonstrate this new principle through an initial hardware prototype based on the single pixel camera.
  arXiv  Detail & Related papers  (2024-07-09T17:17:03Z)
• Neuromorphic Synergy for Video Binarization [54.195375576583864]
  Bimodal objects serve as a visual form to embed information that can be easily recognized by vision systems. Neuromorphic cameras offer new capabilities for alleviating motion blur, but it is non-trivial to first de-blur and then binarize the images in a real-time manner. We propose an event-based binary reconstruction method that leverages the prior knowledge of the bimodal target's properties to perform inference independently in both event space and image space. We also develop an efficient integration method to propagate this binary image to high frame rate binary video.
  arXiv  Detail & Related papers  (2024-02-20T01:43:51Z)
• Real-Time Model-Based Quantitative Ultrasound and Radar [65.268245109828]
  We propose a neural network based on the physical model of wave propagation, which defines the relationship between the received signals and physical properties. Our network can reconstruct multiple physical properties in less than one second for complex and realistic scenarios.
  arXiv  Detail & Related papers  (2024-02-16T09:09:16Z)
• Intelligent Multi-channel Meta-imagers for Accelerating Machine Vision [8.401693241238101]
  We demonstrate an intelligent meta-imager that is designed to work in concert with a digital back-end to off-load computationally expensive convolution operations into high-speed and low-power optics. The meta-imager is employed for object classification, experimentally achieving 98.6% accurate classification of handwritten digits and 88.8% accuracy in classifying fashion images.
  arXiv  Detail & Related papers  (2023-06-12T18:44:08Z)
• Deep learning at the edge enables real-time streaming ptychographic imaging [7.4083593332068975]
  Coherent microscopy techniques like ptychography are poised to revolutionize nanoscale materials characterization. Traditional approaches no longer suffice for recovering sample images in real-time from high-speed coherent imaging experiments. Here, we demonstrate a workflow that leverages artificial intelligence at the edge and high-performance computing to enable real-time inversion on X-ray ptychography data streamed directly from a detector at up to 2 kHz.
  arXiv  Detail & Related papers  (2022-09-20T02:02:37Z)
• Nanoscale Architecture for Frequency-Resolving Single-Photon Detectors [0.0]
  We show that nanoscale elements cooperatively interacting with the photon field in a photodetector architecture allow to simultaneously achieve high efficiency, low jitter, and high frequency resolution. We illustrate the potential performance for frequency resolution over a 1 eV bandwidth in the visible range, indicating near perfect detection efficiency, jitter of a few hundred femtoseconds, and frequency resolution of tens of meV.
  arXiv  Detail & Related papers  (2022-05-12T00:48:54Z)
• 1000x Faster Camera and Machine Vision with Ordinary Devices [76.46540270145698]
  We present vidar, a bit sequence array where each bit represents whether the accumulation of photons has reached a threshold. We have developed a vidar camera that is 1,000x faster than conventional cameras. We have also developed a spiking neural network-based machine vision system that combines the speed of the machine and the mechanism of biological vision.
  arXiv  Detail & Related papers  (2022-01-23T16:10:11Z)
• Photonic neural field on a silicon chip: large-scale, high-speed neuro-inspired computing and sensing [0.0]
  Photonic neural networks have significant potential for high-speed neural processing with low latency and ultralow energy consumption. We propose the concept of a photonic neural field and implement it experimentally on a silicon chip to realize highly scalable neuro-inspired computing. In this study, we use the on-chip photonic neural field as a reservoir of information and demonstrate a high-speed chaotic time-series prediction with low errors.
  arXiv  Detail & Related papers  (2021-05-22T09:28:51Z)
• Photonic reservoir computer based on frequency multiplexing [56.663315405998354]
  We report a photonic implementation of a reservoir computer that exploits frequency domain multiplexing to encode neuron states. The system processes 25 comb lines simultaneously (i.e. 25 neurons), at a rate of 20 MHz.
  arXiv  Detail & Related papers  (2020-08-25T19:30:42Z)
• End-to-end Learning for Inter-Vehicle Distance and Relative Velocity Estimation in ADAS with a Monocular Camera [81.66569124029313]
  We propose a camera-based inter-vehicle distance and relative velocity estimation method based on end-to-end training of a deep neural network. The key novelty of our method is the integration of multiple visual clues provided by any two time-consecutive monocular frames. We also propose a vehicle-centric sampling mechanism to alleviate the effect of perspective distortion in the motion field.
  arXiv  Detail & Related papers  (2020-06-07T08:18:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.