Microsaccade-inspired Event Camera for Robotics

• URL: http://arxiv.org/abs/2405.17769v1
• Date: Tue, 28 May 2024 02:49:46 GMT
• Title: Microsaccade-inspired Event Camera for Robotics
• Authors: Botao He, Ze Wang, Yuan Zhou, Jingxi Chen, Chahat Deep Singh, Haojia Li, Yuman Gao, Shaojie Shen, Kaiwei Wang, Yanjun Cao, Chao Xu, Yiannis Aloimonos, Fei Gao, Cornelia Fermuller,
• Abstract summary: We design an event-based perception system capable of simultaneously maintaining low reaction time and stable texture. The geometrical optics of the rotating wedge prism allows for algorithmic compensation of the additional rotational motion. Various real-world experiments demonstrate the potential of the system to facilitate robotics perception both for low-level and high-level vision tasks.
• Score: 42.27082276343167
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Neuromorphic vision sensors or event cameras have made the visual perception of extremely low reaction time possible, opening new avenues for high-dynamic robotics applications. These event cameras' output is dependent on both motion and texture. However, the event camera fails to capture object edges that are parallel to the camera motion. This is a problem intrinsic to the sensor and therefore challenging to solve algorithmically. Human vision deals with perceptual fading using the active mechanism of small involuntary eye movements, the most prominent ones called microsaccades. By moving the eyes constantly and slightly during fixation, microsaccades can substantially maintain texture stability and persistence. Inspired by microsaccades, we designed an event-based perception system capable of simultaneously maintaining low reaction time and stable texture. In this design, a rotating wedge prism was mounted in front of the aperture of an event camera to redirect light and trigger events. The geometrical optics of the rotating wedge prism allows for algorithmic compensation of the additional rotational motion, resulting in a stable texture appearance and high informational output independent of external motion. The hardware device and software solution are integrated into a system, which we call Artificial MIcrosaccade-enhanced EVent camera (AMI-EV). Benchmark comparisons validate the superior data quality of AMI-EV recordings in scenarios where both standard cameras and event cameras fail to deliver. Various real-world experiments demonstrate the potential of the system to facilitate robotics perception both for low-level and high-level vision tasks.

Related papers

• Deep Event Visual Odometry [40.57142632274148]
  Event cameras offer the exciting possibility of tracking the camera's pose during high-speed motion. Existing event-based monocular visual odometry approaches demonstrate limited performance on recent benchmarks. We present Deep Event VO (DEVO), the first monocular event-only system with strong performance on a large number of real-world benchmarks.
  arXiv  Detail & Related papers  (2023-12-15T14:00:00Z)
• Deep Learning for Event-based Vision: A Comprehensive Survey and Benchmarks [55.81577205593956]
  Event cameras are bio-inspired sensors that capture the per-pixel intensity changes asynchronously. Deep learning (DL) has been brought to this emerging field and inspired active research endeavors in mining its potential.
  arXiv  Detail & Related papers  (2023-02-17T14:19:28Z)
• Lasers to Events: Automatic Extrinsic Calibration of Lidars and Event Cameras [67.84498757689776]
  This paper presents the first direct calibration method between event cameras and lidars. It removes dependencies on frame-based camera intermediaries and/or highly-accurate hand measurements.
  arXiv  Detail & Related papers  (2022-07-03T11:05:45Z)
• TimeReplayer: Unlocking the Potential of Event Cameras for Video Interpolation [78.99283105497489]
  Event camera is a new device to enable video at the presence of arbitrarily complex motion. This paper proposes a novel TimeReplayer algorithm to interpolate videos captured by commodity cameras with events.
  arXiv  Detail & Related papers  (2022-03-25T18:57:42Z)
• A Preliminary Research on Space Situational Awareness Based on Event Cameras [8.27218838055049]
  Event camera is a new type of sensor that is different from traditional cameras. The trigger event is the change of the brightness irradiated on the pixel. Compared with traditional cameras, event cameras have the advantages of high temporal resolution, low latency, high dynamic range, low bandwidth and low power consumption.
  arXiv  Detail & Related papers  (2022-03-24T14:36:18Z)
• ESL: Event-based Structured Light [62.77144631509817]
  Event cameras are bio-inspired sensors providing significant advantages over standard cameras. We propose a novel structured-light system using an event camera to tackle the problem of accurate and high-speed depth sensing.
  arXiv  Detail & Related papers  (2021-11-30T15:47:39Z)
• Moving Object Detection for Event-based Vision using k-means Clustering [0.0]
  Moving object detection is a crucial task in computer vision. Event-based cameras are bio-inspired cameras that work by mimicking the working of the human eye. In this paper, we investigate the application of the k-means clustering technique in detecting moving objects in event-based data.
  arXiv  Detail & Related papers  (2021-09-04T14:43:14Z)
• TUM-VIE: The TUM Stereo Visual-Inertial Event Dataset [50.8779574716494]
  Event cameras are bio-inspired vision sensors which measure per pixel brightness changes. They offer numerous benefits over traditional, frame-based cameras, including low latency, high dynamic range, high temporal resolution and low power consumption. To foster the development of 3D perception and navigation algorithms with event cameras, we present the TUM-VIE dataset.
  arXiv  Detail & Related papers  (2021-08-16T19:53:56Z)
• They See Me Rollin': Inherent Vulnerability of the Rolling Shutter in CMOS Image Sensors [21.5487020124302]
  A camera's electronic rolling shutter can be exploited to inject fine-grained image disruptions. We show how an adversary can modulate the laser to hide up to 75% of objects perceived by state-of-the-art detectors. Our results indicate that rolling shutter attacks can substantially reduce the performance and reliability of vision-based intelligent systems.
  arXiv  Detail & Related papers  (2021-01-25T11:14:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.