Wandering around: A bioinspired approach to visual attention through object motion sensitivity

• URL: http://arxiv.org/abs/2502.06747v1
• Date: Mon, 10 Feb 2025 18:16:30 GMT
• Title: Wandering around: A bioinspired approach to visual attention through object motion sensitivity
• Authors: Giulia D Angelo, Victoria Clerico, Chiara Bartolozzi, Matej Hoffmann, P. Michael Furlong, Alexander Hadjiivanov,
• Abstract summary: Active vision enables dynamic visual perception, offering an alternative to static feedforward architectures in computer vision. Event-based cameras, inspired by the mammalian retina, enhance this capability by capturing asynchronous scene changes. To distinguish moving objects while the event-based camera is in motion the agent requires an object motion segmentation mechanism. This work presents a Convolutional Neural Network bio-inspired attention system for selective attention through object motion sensitivity.
• Score: 40.966228784674115
• License:
• Abstract: Active vision enables dynamic visual perception, offering an alternative to static feedforward architectures in computer vision, which rely on large datasets and high computational resources. Biological selective attention mechanisms allow agents to focus on salient Regions of Interest (ROIs), reducing computational demand while maintaining real-time responsiveness. Event-based cameras, inspired by the mammalian retina, enhance this capability by capturing asynchronous scene changes enabling efficient low-latency processing. To distinguish moving objects while the event-based camera is in motion the agent requires an object motion segmentation mechanism to accurately detect targets and center them in the visual field (fovea). Integrating event-based sensors with neuromorphic algorithms represents a paradigm shift, using Spiking Neural Networks to parallelize computation and adapt to dynamic environments. This work presents a Spiking Convolutional Neural Network bioinspired attention system for selective attention through object motion sensitivity. The system generates events via fixational eye movements using a Dynamic Vision Sensor integrated into the Speck neuromorphic hardware, mounted on a Pan-Tilt unit, to identify the ROI and saccade toward it. The system, characterized using ideal gratings and benchmarked against the Event Camera Motion Segmentation Dataset, reaches a mean IoU of 82.2% and a mean SSIM of 96% in multi-object motion segmentation. The detection of salient objects reaches 88.8% accuracy in office scenarios and 89.8% in low-light conditions on the Event-Assisted Low-Light Video Object Segmentation Dataset. A real-time demonstrator shows the system's 0.12 s response to dynamic scenes. Its learning-free design ensures robustness across perceptual scenes, making it a reliable foundation for real-time robotic applications serving as a basis for more complex architectures.

Related papers

• Deep Learning and Hybrid Approaches for Dynamic Scene Analysis, Object Detection and Motion Tracking [0.0]
  This project aims to develop a robust video surveillance system, which can segment videos into smaller clips based on the detection of activities. It uses CCTV footage, for example, to record only major events-like the appearance of a person or a thief-so that storage is optimized and digital searches are easier.
  arXiv  Detail & Related papers  (2024-12-05T07:44:40Z)
• Retina-Inspired Object Motion Segmentation for Event-Cameras [0.0]
  Event-cameras have emerged as a revolutionary technology with a high temporal resolution that far surpasses standard active pixel cameras. This research showcases the potential of additional retinal functionalities to extract visual features.
  arXiv  Detail & Related papers  (2024-08-18T12:28:26Z)
• Motion Segmentation for Neuromorphic Aerial Surveillance [42.04157319642197]
  Event cameras offer superior temporal resolution, superior dynamic range, and minimal power requirements. Unlike traditional frame-based sensors that capture redundant information at fixed intervals, event cameras asynchronously record pixel-level brightness changes. We introduce a novel motion segmentation method that leverages self-supervised vision transformers on both event data and optical flow information.
  arXiv  Detail & Related papers  (2024-05-24T04:36:13Z)
• Neuromorphic Vision-based Motion Segmentation with Graph Transformer Neural Network [4.386534439007928]
  We propose a novel event-based motion segmentation algorithm using a Graph Transformer Neural Network, dubbed GTNN. Our proposed algorithm processes event streams as 3D graphs by a series nonlinear transformations to unveil local and global correlations between events. We show that GTNN outperforms state-of-the-art methods in the presence of dynamic background variations, motion patterns, and multiple dynamic objects with varying sizes and velocities.
  arXiv  Detail & Related papers  (2024-04-16T22:44:29Z)
• SpikeMOT: Event-based Multi-Object Tracking with Sparse Motion Features [52.213656737672935]
  SpikeMOT is an event-based multi-object tracker. SpikeMOT uses spiking neural networks to extract sparsetemporal features from event streams associated with objects.
  arXiv  Detail & Related papers  (2023-09-29T05:13:43Z)
• DynaMoN: Motion-Aware Fast and Robust Camera Localization for Dynamic Neural Radiance Fields [71.94156412354054]
  We propose Dynamic Motion-Aware Fast and Robust Camera Localization for Dynamic Neural Radiance Fields (DynaMoN) DynaMoN handles dynamic content for initial camera pose estimation and statics-focused ray sampling for fast and accurate novel-view synthesis. We extensively evaluate our approach on two real-world dynamic datasets, the TUM RGB-D dataset and the BONN RGB-D Dynamic dataset.
  arXiv  Detail & Related papers  (2023-09-16T08:46:59Z)
• MotionTrack: Learning Motion Predictor for Multiple Object Tracking [68.68339102749358]
  We introduce a novel motion-based tracker, MotionTrack, centered around a learnable motion predictor. Our experimental results demonstrate that MotionTrack yields state-of-the-art performance on datasets such as Dancetrack and SportsMOT.
  arXiv  Detail & Related papers  (2023-06-05T04:24:11Z)
• Alignment-free HDR Deghosting with Semantics Consistent Transformer [76.91669741684173]
  High dynamic range imaging aims to retrieve information from multiple low-dynamic range inputs to generate realistic output. Existing methods often focus on the spatial misalignment across input frames caused by the foreground and/or camera motion. We propose a novel alignment-free network with a Semantics Consistent Transformer (SCTNet) with both spatial and channel attention modules.
  arXiv  Detail & Related papers  (2023-05-29T15:03:23Z)
• Event-based Motion Segmentation with Spatio-Temporal Graph Cuts [51.17064599766138]
  We have developed a method to identify independently objects acquired with an event-based camera. The method performs on par or better than the state of the art without having to predetermine the number of expected moving objects.
  arXiv  Detail & Related papers  (2020-12-16T04:06:02Z)
• Neuromorphic Eye-in-Hand Visual Servoing [0.9949801888214528]
  Event cameras give human-like vision capabilities with low latency and wide dynamic range. We present a visual servoing method using an event camera and a switching control strategy to explore, reach and grasp. Experiments prove the effectiveness of the method to track and grasp objects of different shapes without the need for re-tuning.
  arXiv  Detail & Related papers  (2020-04-15T23:57:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.