Programmable Silicon Retina on Pixel Processor Array

• URL: http://arxiv.org/abs/2606.08370v1
• Date: Sat, 06 Jun 2026 23:10:48 GMT
• Title: Programmable Silicon Retina on Pixel Processor Array
• Abstract summary: We present the first implementation of a multi-stage Silicon Retina model on the SCAMP-5 Pixel Processor Array.<n>We evaluate the performance of our model on Video Intensity Reconstruction and Video Saliency Prediction.<n>Results suggest that the silicon retina's "information distillation" mechanism can achieve a more efficient representation for downstream neural networks.
• Score: 42.80978894922492
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Standard dynamic vision sensors approximate retinal processing by detecting temporal contrast changes, offering high speed and high dynamic range. In this work, we explore whether incorporating additional biologically inspired processing stages - specifically spatial filtering and gain control - can offer advantages for certain downstream tasks such as saliency prediction. We present the first implementation of a multi-stage Silicon Retina model on the SCAMP-5 Pixel Processor Array, along with a GPU-based simulation framework. We evaluate the performance of our model on Video Intensity Reconstruction and Video Saliency Prediction. While the bio-inspired model is less effective at reconstructing absolute intensity frames, it achieves a 13\% reduction in saliency prediction loss in comparison to standard DVS event representation, while reducing the event rate by approximately 47\%. These experiments are obtained using a lightweight $\approx 100$k-parameter FireNet-style network, adapted from event-based reconstruction to saliency prediction. These results suggest that the silicon retina's "information distillation" mechanism can achieve a more efficient representation for downstream neural networks, particularly in bandwidth-constrained edge applications.

Related papers

• $D^3$-RSMDE: 40$\times$ Faster and High-Fidelity Remote Sensing Monocular Depth Estimation [72.9912717963138]
  Real-time, high-fidelity monocular depth estimation from remote sensing imagery is crucial for numerous applications.<n>Although using Vision Transformer (ViT) backbones for dense prediction is fast, they often exhibit poor perceptual quality.<n>We propose Depth Detail Diffusion for Remote Sensing Monocular Depth Estimation ($D3$-RSMDE) to overcome these limitations.<n>$D3$-RSMDE achieves a notable 11.85% reduction in the Learned Perceptual Image Patch Similarity (LPIPS) perceptual metric.
  arXiv  Detail & Related papers  (2026-03-17T10:50:36Z)
• Enhancing Eye Feature Estimation from Event Data Streams through Adaptive Inference State Space Modeling [68.1289208938377]
  Eye feature extraction from event-based data streams can be performed efficiently and with low energy consumption.<n>We introduce the emphadaptive inference state space model (AISSM), a novel architecture for feature extraction.<n>We also craft and evaluate a novel learning technique that improves training efficiency.
  arXiv  Detail & Related papers  (2026-03-14T18:47:08Z)
• Dynamic Graph Induced Contour-aware Heat Conduction Network for Event-based Object Detection [42.021851148914145]
  Event-based Vision Sensors (EVS) have demonstrated significant advantages over traditional RGB frame-based cameras in low-light conditions.<n>This paper proposes a novel dynamic graph induced contour-aware heat conduction network for event stream based object detection.
  arXiv  Detail & Related papers  (2025-05-19T09:44:01Z)
• Task-Oriented Real-time Visual Inference for IoVT Systems: A Co-design Framework of Neural Networks and Edge Deployment [61.20689382879937]
  Task-oriented edge computing addresses this by shifting data analysis to the edge. Existing methods struggle to balance high model performance with low resource consumption. We propose a novel co-design framework to optimize neural network architecture.
  arXiv  Detail & Related papers  (2024-10-29T19:02:54Z)
• Optimal OnTheFly Feedback Control of Event Sensors [0.14999444543328289]
  Event-based vision sensors produce an asynchronous stream of events which are triggered when pixel intensity variation exceeds a threshold. We propose an approach for dynamic feedback control of activation thresholds, in which a controller network analyzes the past emitted events. We demonstrate that our approach outperforms both fixed and randomly-varying threshold schemes by 6-12% in terms of LPIPS perceptual image dissimilarity metric.
  arXiv  Detail & Related papers  (2024-08-23T10:49:16Z)
• Finding Visual Saliency in Continuous Spike Stream [23.591309376586835]
  In this paper, we investigate the visual saliency in the continuous spike stream for the first time. We propose a Recurrent Spiking Transformer framework, which is based on a full spiking neural network. Our framework exhibits a substantial margin of improvement in highlighting and capturing visual saliency in the spike stream.
  arXiv  Detail & Related papers  (2024-03-10T15:15:35Z)
• DGNet: Dynamic Gradient-Guided Network for Water-Related Optics Image Enhancement [77.0360085530701]
  Underwater image enhancement (UIE) is a challenging task due to the complex degradation caused by underwater environments. Previous methods often idealize the degradation process, and neglect the impact of medium noise and object motion on the distribution of image features. Our approach utilizes predicted images to dynamically update pseudo-labels, adding a dynamic gradient to optimize the network's gradient space.
  arXiv  Detail & Related papers  (2023-12-12T06:07:21Z)
• SpikeSEE: An Energy-Efficient Dynamic Scenes Processing Framework for Retinal Prostheses [3.794154439461156]
  We propose an energy-efficient dynamic scenes processing framework (SpikeSEE) that combines a spike representation encoding technique and a bio-inspired spiking recurrent neural network (SRNN) model. Our proposed SpikeSEE predicts the response of ganglion cells more accurately with lower energy consumption.
  arXiv  Detail & Related papers  (2022-09-16T12:46:10Z)
• Learning Monocular Dense Depth from Events [53.078665310545745]
  Event cameras produce brightness changes in the form of a stream of asynchronous events instead of intensity frames. Recent learning-based approaches have been applied to event-based data, such as monocular depth prediction. We propose a recurrent architecture to solve this task and show significant improvement over standard feed-forward methods.
  arXiv  Detail & Related papers  (2020-10-16T12:36:23Z)
• Optimisation of a Siamese Neural Network for Real-Time Energy Efficient Object Tracking [0.0]
  optimisation of visual object tracking using a Siamese neural network for embedded vision systems is presented. It was assumed that the solution shall operate in real-time, preferably for a high resolution video stream.
  arXiv  Detail & Related papers  (2020-07-01T13:49:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.