Quantitative evaluation of brain-inspired vision sensors in high-speed robotic perception

• URL: http://arxiv.org/abs/2504.19253v1
• Date: Sun, 27 Apr 2025 14:24:06 GMT
• Title: Quantitative evaluation of brain-inspired vision sensors in high-speed robotic perception
• Abstract summary: Perception systems in robotics encounter significant challenges in high-speed and dynamic conditions when relying on traditional cameras.<n>Brain-inspired vision sensors (BVS) have recently gained attention as an alternative, offering high temporal resolution with reduced bandwidth and power.<n>Here, we present the first quantitative evaluation framework for two representative classes of BVSs in variable-speed robotic sensing.
• Score: 4.9554369813645325
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Perception systems in robotics encounter significant challenges in high-speed and dynamic conditions when relying on traditional cameras, where motion blur can compromise spatial feature integrity and task performance. Brain-inspired vision sensors (BVS) have recently gained attention as an alternative, offering high temporal resolution with reduced bandwidth and power requirements. Here, we present the first quantitative evaluation framework for two representative classes of BVSs in variable-speed robotic sensing, including event-based vision sensors (EVS) that detect asynchronous temporal contrasts, and the primitive-based sensor Tianmouc that employs a complementary mechanism to encode both spatiotemporal changes and intensity. A unified testing protocol is established, including crosssensor calibrations, standardized testing platforms, and quality metrics to address differences in data modality. From an imaging standpoint, we evaluate the effects of sensor non-idealities, such as motion-induced distortion, on the capture of structural information. For functional benchmarking, we examine task performance in corner detection and motion estimation under different rotational speeds. Results indicate that EVS performs well in highspeed, sparse scenarios and in modestly fast, complex scenes, but exhibits performance limitations in high-speed, cluttered settings due to pixel-level bandwidth variations and event rate saturation. In comparison, Tianmouc demonstrates consistent performance across sparse and complex scenarios at various speeds, supported by its global, precise, high-speed spatiotemporal gradient samplings. These findings offer valuable insights into the applicationdependent suitability of BVS technologies and support further advancement in this area.

Related papers

• Sen-Cap: Sensor-Flexible and Noise-Resilient Human Motion Capture via LiDAR-Camera Integration [50.18488308004802]
  Sen-Cap is a 3D motion capture framework that integrates multi-modal data from LiDAR and camera.<n>Sen-Cap achieves state-of-the-art performance on major metrics on Human-M3 and FreeMotion.<n>This combination of flexibility and robustness opens new opportunities for motion capture in real-world scenarios.
  arXiv  Detail & Related papers  (2026-08-03T14:16:18Z)
• WiFuse: An Attention Mechanism for Human Activity Recognition using Fused CSI Amplitude and Delay-Doppler Channel Features [1.0554048699217669]
  WiFuse is a dual-stream Channel State Information framework for human activity recognition.<n>It pairs amplitude and Delay-Doppler representations within a dual-stream strategy, enhanced by transfer learning.<n>It reaches an overall accuracy of up to 95.28% across the four environments of the XRF55 dataset and up to 98.20% on the multi-user Wi-MIR dataset.
  arXiv  Detail & Related papers  (2026-08-01T12:54:09Z)
• Sensor Configuration Matters: A Systematic Evaluation of Multimodal SLAM on Quadruped Robots [15.924751226144979]
  We present a systematic evaluation of state-of-the-art visual, visual-inertial, and LiDAR-visual-inertial SLAM methods.<n>We quantify the impacts of camera modalities, shutter techniques, and inertial sensor tiers, analyzing their trade-offs across localization accuracy, algorithmic robustness, and computational resource utilization.
  arXiv  Detail & Related papers  (2026-06-17T13:41:07Z)
• Event-based SLAM Benchmark for High-Speed Maneuvers [68.148886127117]
  Event-based cameras are bio-inspired sensors with pixels that independently and asynchronously respond to brightness changes at microsecond resolution.<n>Existing event-based approaches, although successful in mitigating motion blur caused by high-speed maneuvers, suffer from many limitations.<n>We introduce a benchmarking framework for event-based state estimation, called EvSLAM, characterized by sufficient variation in data collection platforms.
  arXiv  Detail & Related papers  (2026-04-27T04:36:41Z)
• Hypergraph-State Collaborative Reasoning for Multi-Object Tracking [63.32950991964095]
  We propose a collaborative reasoning framework that enhances motion estimation through joint inference among multiple correlated objects.<n>By allowing objects with similar motion states to mutually constrain and refine each other, our framework stabilizes noisy trajectories and infers plausible motion continuity even when target is occluded.
  arXiv  Detail & Related papers  (2026-04-14T12:37:39Z)
• SynSacc: A Blender-to-V2E Pipeline for Synthetic Neuromorphic Eye-Movement Data and Sim-to-Real Spiking Model Training [6.113940256355538]
  We introduce a synthetic dataset generated with Blender to simulate saccades and fixations under controlled conditions.<n>We evaluate its robustness by training two architectures and finetuning on real event data.<n>The proposed models achieve up to 0.83 accuracy and maintain consistent performance across varying temporal resolutions.
  arXiv  Detail & Related papers  (2026-02-09T14:34:31Z)
• HAD: Hierarchical Asymmetric Distillation to Bridge Spatio-Temporal Gaps in Event-Based Object Tracking [80.07224739976911]
  Event cameras offer exceptional temporal resolution and a range (modal)<n> RGB cameras excel at capturing rich texture with high resolution, whereas event cameras offer exceptional temporal resolution and a range (modal)
  arXiv  Detail & Related papers  (2025-10-22T13:15:13Z)
• MIORe & VAR-MIORe: Benchmarks to Push the Boundaries of Restoration [53.180212987726556]
  We introduce MIORe and VAR-MIORe, two novel multi-task datasets that address critical limitations in current motion restoration benchmarks.<n>Our datasets capture a broad spectrum of motion scenarios, which include complex ego-camera movements, dynamic multi-subject interactions, and depth-dependent blur effects.
  arXiv  Detail & Related papers  (2025-09-08T15:34:31Z)
• Progressive Inertial Poser: Progressive Real-Time Kinematic Chain Estimation for 3D Full-Body Pose from Three IMU Sensors [25.67875816218477]
  Full-body pose estimation from sparse tracking signals is not limited by environmental conditions or recording range.<n>Previous works either face the challenge of wearing additional sensors on the pelvis and lower-body or rely on external visual sensors to obtain global positions of key joints.<n>To improve the practicality of the technology for virtual reality applications, we estimate full-body poses using only inertial data obtained from three Inertial Measurement Unit (IMU) sensors worn on the head and wrists.
  arXiv  Detail & Related papers  (2025-05-08T15:28:09Z)
• Event-Based Tracking Any Point with Motion-Augmented Temporal Consistency [58.719310295870024]
  This paper presents an event-based framework for tracking any point.<n>It tackles the challenges posed by spatial sparsity and motion sensitivity in events.<n>It achieves 150% faster processing with competitive model parameters.
  arXiv  Detail & Related papers  (2024-12-02T09:13:29Z)
• Descriptor: Face Detection Dataset for Programmable Threshold-Based Sparse-Vision [0.8271394038014485]
  This dataset is an annotated, temporal-threshold-based vision dataset for face detection tasks derived from the same videos used for Aff-Wild2. We anticipate that this resource will significantly support the development of robust vision systems based on smart sensors that can process based on temporal-difference thresholds.
  arXiv  Detail & Related papers  (2024-10-01T03:42:03Z)
• DemosaicFormer: Coarse-to-Fine Demosaicing Network for HybridEVS Camera [70.28702677370879]
  Hybrid Event-Based Vision Sensor (HybridEVS) is a novel sensor integrating traditional frame-based and event-based sensors. Despite its potential, the lack of Image signal processing (ISP) pipeline specifically designed for HybridEVS poses a significant challenge. We propose a coarse-to-fine framework named DemosaicFormer which comprises coarse demosaicing and pixel correction.
  arXiv  Detail & Related papers  (2024-06-12T07:20:46Z)
• Motion Segmentation for Neuromorphic Aerial Surveillance [42.04157319642197]
  Event cameras offer superior temporal resolution, superior dynamic range, and minimal power requirements. Unlike traditional frame-based sensors that capture redundant information at fixed intervals, event cameras asynchronously record pixel-level brightness changes. We introduce a novel motion segmentation method that leverages self-supervised vision transformers on both event data and optical flow information.
  arXiv  Detail & Related papers  (2024-05-24T04:36:13Z)
• V2CE: Video to Continuous Events Simulator [1.1009908861287052]
  We present a novel method for video-to-events stream conversion from multiple perspectives, considering the specific characteristics of Dynamic Vision Sensor (DVS) A series of carefully designed timestamp losses helps enhance the quality of generated event voxels significantly. We also propose a novel local dynamic-aware inference strategy to accurately recover event timestamps from event voxels in a continuous fashion.
  arXiv  Detail & Related papers  (2023-09-16T06:06:53Z)
• EventTransAct: A video transformer-based framework for Event-camera based action recognition [52.537021302246664]
  Event cameras offer new opportunities compared to standard action recognition in RGB videos. In this study, we employ a computationally efficient model, namely the video transformer network (VTN), which initially acquires spatial embeddings per event-frame. In order to better adopt the VTN for the sparse and fine-grained nature of event data, we design Event-Contrastive Loss ($mathcalL_EC$) and event-specific augmentations.
  arXiv  Detail & Related papers  (2023-08-25T23:51:07Z)
• Event-based Simultaneous Localization and Mapping: A Comprehensive Survey [52.73728442921428]
  Review of event-based vSLAM algorithms that exploit the benefits of asynchronous and irregular event streams for localization and mapping tasks. Paper categorizes event-based vSLAM methods into four main categories: feature-based, direct, motion-compensation, and deep learning methods.
  arXiv  Detail & Related papers  (2023-04-19T16:21:14Z)
• Recurrent Vision Transformers for Object Detection with Event Cameras [62.27246562304705]
  We present Recurrent Vision Transformers (RVTs), a novel backbone for object detection with event cameras. RVTs can be trained from scratch to reach state-of-the-art performance on event-based object detection. Our study brings new insights into effective design choices that can be fruitful for research beyond event-based vision.
  arXiv  Detail & Related papers  (2022-12-11T20:28:59Z)
• High Speed Rotation Estimation with Dynamic Vision Sensors [10.394670846430635]
  The Relative Mean Absolute Error (RMAE) of EV-Tach is as low as 0.03% which is comparable to the state-of-the-art laser under fixed measurement mode. EV-Tach is robust to subtle movement of user's hand, therefore, can be used as a handheld device, where the laser fails to produce reasonable results.
  arXiv  Detail & Related papers  (2022-09-06T04:00:46Z)
• EBBINNOT: A Hardware Efficient Hybrid Event-Frame Tracker for Stationary Dynamic Vision Sensors [5.674895233111088]
  This paper presents a hybrid event-frame approach for detecting and tracking objects recorded by a stationary neuromorphic sensor. To exploit the background removal property of a static DVS, we propose an event-based binary image creation that signals presence or absence of events in a frame duration. This is the first time a stationary DVS based traffic monitoring solution is extensively compared to simultaneously recorded RGB frame-based methods.
  arXiv  Detail & Related papers  (2020-05-31T03:01:35Z)
• Event-based Asynchronous Sparse Convolutional Networks [54.094244806123235]
  Event cameras are bio-inspired sensors that respond to per-pixel brightness changes in the form of asynchronous and sparse "events" We present a general framework for converting models trained on synchronous image-like event representations into asynchronous models with identical output. We show both theoretically and experimentally that this drastically reduces the computational complexity and latency of high-capacity, synchronous neural networks.
  arXiv  Detail & Related papers  (2020-03-20T08:39:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.