Instantaneous Stereo Depth Estimation of Real-World Stimuli with a Neuromorphic Stereo-Vision Setup

• URL: http://arxiv.org/abs/2104.02541v1
• Date: Tue, 6 Apr 2021 14:31:23 GMT
• Title: Instantaneous Stereo Depth Estimation of Real-World Stimuli with a Neuromorphic Stereo-Vision Setup
• Authors: Nicoletta Risi, Enrico Calabrese, Giacomo Indiveri
• Abstract summary: Spiking Neural Network (SNN) architectures for stereo vision have the potential of simplifying the stereo-matching problem. We validate a brain-inspired event-based stereo-matching architecture implemented on a mixed-signal neuromorphic processor with real-world data.
• Score: 4.28479274054892
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The stereo-matching problem, i.e., matching corresponding features in two different views to reconstruct depth, is efficiently solved in biology. Yet, it remains the computational bottleneck for classical machine vision approaches. By exploiting the properties of event cameras, recently proposed Spiking Neural Network (SNN) architectures for stereo vision have the potential of simplifying the stereo-matching problem. Several solutions that combine event cameras with spike-based neuromorphic processors already exist. However, they are either simulated on digital hardware or tested on simplified stimuli. In this work, we use the Dynamic Vision Sensor 3D Human Pose Dataset (DHP19) to validate a brain-inspired event-based stereo-matching architecture implemented on a mixed-signal neuromorphic processor with real-world data. Our experiments show that this SNN architecture, composed of coincidence detectors and disparity sensitive neurons, is able to provide a coarse estimate of the input disparity instantaneously, thereby detecting the presence of a stimulus moving in depth in real-time.

Related papers

• Brain3D: Generating 3D Objects from fMRI [76.41771117405973]
  We design a novel 3D object representation learning method, Brain3D, that takes as input the fMRI data of a subject. We show that our model captures the distinct functionalities of each region of human vision system. Preliminary evaluations indicate that Brain3D can successfully identify the disordered brain regions in simulated scenarios.
  arXiv  Detail & Related papers  (2024-05-24T06:06:11Z)
• MinD-3D: Reconstruct High-quality 3D objects in Human Brain [50.534007259536715]
  Recon3DMind is an innovative task aimed at reconstructing 3D visuals from Functional Magnetic Resonance Imaging (fMRI) signals. We present the fMRI-Shape dataset, which includes data from 14 participants and features 360-degree videos of 3D objects. We propose MinD-3D, a novel and effective three-stage framework specifically designed to decode the brain's 3D visual information from fMRI signals.
  arXiv  Detail & Related papers  (2023-12-12T18:21:36Z)
• Hopfield-Enhanced Deep Neural Networks for Artifact-Resilient Brain State Decoding [0.0]
  We propose a two-stage computational framework combining Hopfield Networks for artifact data preprocessing with Conal Neural Networks (CNNs) for classification of brain states in rat neural recordings under different levels of anesthesia. Performance across various levels of data compression and noise intensities showed that our framework can effectively mitigate artifacts, allowing the model to reach parity with the clean-data CNN at lower noise levels.
  arXiv  Detail & Related papers  (2023-11-06T15:08:13Z)
• Probing neural representations of scene perception in a hippocampally dependent task using artificial neural networks [1.0312968200748116]
  Deep artificial neural networks (DNNs) trained through backpropagation provide effective models of the mammalian visual system. We describe a novel scene perception benchmark inspired by a hippocampal dependent task. Using a network architecture inspired by the connectivity between temporal lobe structures and the hippocampus, we demonstrate that DNNs trained using a triplet loss can learn this task.
  arXiv  Detail & Related papers  (2023-03-11T10:26:25Z)
• StereoVoxelNet: Real-Time Obstacle Detection Based on Occupancy Voxels from a Stereo Camera Using Deep Neural Networks [32.7826524859756]
  Obstacle detection is a safety-critical problem in robot navigation, where stereo matching is a popular vision-based approach. This paper proposes a computationally efficient method that leverages a deep neural network to detect occupancy from stereo images directly. Our approach detects obstacles accurately in the range of 32 meters and achieves better IoU (Intersection over Union) and CD (Chamfer Distance) scores with only 2% of the computation cost of the state-of-the-art stereo model.
  arXiv  Detail & Related papers  (2022-09-18T03:32:38Z)
• Visual Odometry with Neuromorphic Resonator Networks [9.903137966539898]
  Visual Odometry (VO) is a method to estimate self-motion of a mobile robot using visual sensors. Neuromorphic hardware offers low-power solutions to many vision and AI problems. We present a modular neuromorphic algorithm that achieves state-of-the-art performance on two-dimensional VO tasks.
  arXiv  Detail & Related papers  (2022-09-05T14:57:03Z)
• Neural Disparity Refinement for Arbitrary Resolution Stereo [67.55946402652778]
  We introduce a novel architecture for neural disparity refinement aimed at facilitating deployment of 3D computer vision on cheap and widespread consumer devices. Our approach relies on a continuous formulation that enables to estimate a refined disparity map at any arbitrary output resolution.
  arXiv  Detail & Related papers  (2021-10-28T18:00:00Z)
• StereoSpike: Depth Learning with a Spiking Neural Network [0.0]
  We present an end-to-end neuromorphic approach to depth estimation. We use a Spiking Neural Network (SNN) with a slightly modified U-Net-like encoder-decoder architecture, that we named StereoSpike. We demonstrate that this architecture generalizes very well, even better than its non-spiking counterparts.
  arXiv  Detail & Related papers  (2021-09-28T14:11:36Z)
• Continuous Emotion Recognition with Spatiotemporal Convolutional Neural Networks [82.54695985117783]
  We investigate the suitability of state-of-the-art deep learning architectures for continuous emotion recognition using long video sequences captured in-the-wild. We have developed and evaluated convolutional recurrent neural networks combining 2D-CNNs and long short term-memory units, and inflated 3D-CNN models, which are built by inflating the weights of a pre-trained 2D-CNN model during fine-tuning.
  arXiv  Detail & Related papers  (2020-11-18T13:42:05Z)
• Multi-Tones' Phase Coding (MTPC) of Interaural Time Difference by Spiking Neural Network [68.43026108936029]
  We propose a pure spiking neural network (SNN) based computational model for precise sound localization in the noisy real-world environment. We implement this algorithm in a real-time robotic system with a microphone array. The experiment results show a mean error azimuth of 13 degrees, which surpasses the accuracy of the other biologically plausible neuromorphic approach for sound source localization.
  arXiv  Detail & Related papers  (2020-07-07T08:22:56Z)
• 4D Spatio-Temporal Convolutional Networks for Object Position Estimation in OCT Volumes [69.62333053044712]
  3D convolutional neural networks (CNNs) have shown promising performance for pose estimation of a marker object using single OCT images. We extend 3D CNNs to 4D-temporal CNNs to evaluate the impact of additional temporal information for marker object tracking.
  arXiv  Detail & Related papers  (2020-07-02T12:02:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.