1000x Faster Camera and Machine Vision with Ordinary Devices
- URL: http://arxiv.org/abs/2201.09302v1
- Date: Sun, 23 Jan 2022 16:10:11 GMT
- Title: 1000x Faster Camera and Machine Vision with Ordinary Devices
- Authors: Tiejun Huang, Yajing Zheng, Zhaofei Yu, Rui Chen, Yuan Li, Ruiqin
Xiong, Lei Ma, Junwei Zhao, Siwei Dong, Lin Zhu, Jianing Li, Shanshan Jia,
Yihua Fu, Boxin Shi, Si Wu and Yonghong Tian
- Abstract summary: We present vidar, a bit sequence array where each bit represents whether the accumulation of photons has reached a threshold.
We have developed a vidar camera that is 1,000x faster than conventional cameras.
We have also developed a spiking neural network-based machine vision system that combines the speed of the machine and the mechanism of biological vision.
- Score: 76.46540270145698
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In digital cameras, we find a major limitation: the image and video form
inherited from a film camera obstructs it from capturing the rapidly changing
photonic world. Here, we present vidar, a bit sequence array where each bit
represents whether the accumulation of photons has reached a threshold, to
record and reconstruct the scene radiance at any moment. By employing only
consumer-level CMOS sensors and integrated circuits, we have developed a vidar
camera that is 1,000x faster than conventional cameras. By treating vidar as
spike trains in biological vision, we have further developed a spiking neural
network-based machine vision system that combines the speed of the machine and
the mechanism of biological vision, achieving high-speed object detection and
tracking 1,000x faster than human vision. We demonstrate the utility of the
vidar camera and the super vision system in an assistant referee and target
pointing system. Our study is expected to fundamentally revolutionize the image
and video concepts and related industries, including photography, movies, and
visual media, and to unseal a new spiking neural network-enabled speed-free
machine vision era.
Related papers
- Motion Capture from Inertial and Vision Sensors [60.5190090684795]
MINIONS is a large-scale Motion capture dataset collected from INertial and visION Sensors.
We conduct experiments on multi-modal motion capture using a monocular camera and very few IMUs.
arXiv Detail & Related papers (2024-07-23T09:41:10Z) - Microsaccade-inspired Event Camera for Robotics [42.27082276343167]
We design an event-based perception system capable of simultaneously maintaining low reaction time and stable texture.
The geometrical optics of the rotating wedge prism allows for algorithmic compensation of the additional rotational motion.
Various real-world experiments demonstrate the potential of the system to facilitate robotics perception both for low-level and high-level vision tasks.
arXiv Detail & Related papers (2024-05-28T02:49:46Z) - Neuromorphic Synergy for Video Binarization [54.195375576583864]
Bimodal objects serve as a visual form to embed information that can be easily recognized by vision systems.
Neuromorphic cameras offer new capabilities for alleviating motion blur, but it is non-trivial to first de-blur and then binarize the images in a real-time manner.
We propose an event-based binary reconstruction method that leverages the prior knowledge of the bimodal target's properties to perform inference independently in both event space and image space.
We also develop an efficient integration method to propagate this binary image to high frame rate binary video.
arXiv Detail & Related papers (2024-02-20T01:43:51Z) - Inverting the Imaging Process by Learning an Implicit Camera Model [73.81635386829846]
This paper proposes a novel implicit camera model which represents the physical imaging process of a camera as a deep neural network.
We demonstrate the power of this new implicit camera model on two inverse imaging tasks.
arXiv Detail & Related papers (2023-04-25T11:55:03Z) - Deep Learning for Event-based Vision: A Comprehensive Survey and Benchmarks [55.81577205593956]
Event cameras are bio-inspired sensors that capture the per-pixel intensity changes asynchronously.
Deep learning (DL) has been brought to this emerging field and inspired active research endeavors in mining its potential.
arXiv Detail & Related papers (2023-02-17T14:19:28Z) - Ultrafast single-channel machine vision based on neuro-inspired photonic
computing [0.0]
Neuro-inspired photonic computing is a promising approach to speed-up machine vision processing with ultralow latency.
Here, we propose an image-sensor-free machine vision framework, which optically processes real-world visual information with only a single input channel.
We experimentally demonstrate that the proposed approach is capable of high-speed image recognition and anomaly detection, and furthermore, it can be used for high-speed imaging.
arXiv Detail & Related papers (2023-02-15T10:08:04Z) - EDeNN: Event Decay Neural Networks for low latency vision [26.784944204163363]
We develop a new type of neural network which operates closer to the original event data stream.
We demonstrate state-of-the-art performance in angular velocity regression and competitive optical flow estimation.
arXiv Detail & Related papers (2022-09-09T15:51:39Z) - TUM-VIE: The TUM Stereo Visual-Inertial Event Dataset [50.8779574716494]
Event cameras are bio-inspired vision sensors which measure per pixel brightness changes.
They offer numerous benefits over traditional, frame-based cameras, including low latency, high dynamic range, high temporal resolution and low power consumption.
To foster the development of 3D perception and navigation algorithms with event cameras, we present the TUM-VIE dataset.
arXiv Detail & Related papers (2021-08-16T19:53:56Z) - Real-Time Face & Eye Tracking and Blink Detection using Event Cameras [3.842206880015537]
Event cameras contain emerging, neuromorphic vision sensors that capture local light intensity changes at each pixel, generating a stream of asynchronous events.
Driver monitoring systems (DMS) are in-cabin safety systems designed to sense and understand a drivers physical and cognitive state.
This paper proposes a novel method to simultaneously detect and track faces and eyes for driver monitoring.
arXiv Detail & Related papers (2020-10-16T10:02:41Z) - Performance Evaluation of Low-Cost Machine Vision Cameras for
Image-Based Grasp Verification [0.0]
In this paper, we propose a vision based grasp verification system using machine vision cameras.
Our experiments demonstrate that the selected machine vision camera and the deep learning models can robustly verify grasp with 97% per frame accuracy.
arXiv Detail & Related papers (2020-03-23T10:34:27Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.