Retina-Inspired Object Motion Segmentation for Event-Cameras

• URL: http://arxiv.org/abs/2408.09454v2
• Date: Fri, 06 Dec 2024 12:37:08 GMT
• Title: Retina-Inspired Object Motion Segmentation for Event-Cameras
• Authors: Victoria Clerico, Shay Snyder, Arya Lohia, Md Abdullah-Al Kaiser, Gregory Schwartz, Akhilesh Jaiswal, Maryam Parsa,
• Abstract summary: Event-cameras have emerged as a revolutionary technology with a high temporal resolution that far surpasses standard active pixel cameras. This research showcases the potential of additional retinal functionalities to extract visual features.
• Score: 0.0
• License:
• Abstract: Event-cameras have emerged as a revolutionary technology with a high temporal resolution that far surpasses standard active pixel cameras. This technology draws biological inspiration from photoreceptors and the initial retinal synapse. This research showcases the potential of additional retinal functionalities to extract visual features. We provide a domain-agnostic and efficient algorithm for ego-motion compensation based on Object Motion Sensitivity (OMS), one of the multiple features computed within the mammalian retina. We develop a method based on experimental neuroscience that translates OMS' biological circuitry to a low-overhead algorithm to suppress camera motion bypassing the need for deep networks and learning. Our system processes event data from dynamic scenes to perform pixel-wise object motion segmentation using a real and synthetic dataset. This paper introduces a bio-inspired computer vision method that dramatically reduces the number of parameters by $\text{10}^\text{3}$ to $\text{10}^\text{6}$ orders of magnitude compared to previous approaches. Our work paves the way for robust, high-speed, and low-bandwidth decision-making for in-sensor computations.

Related papers

• Wandering around: A bioinspired approach to visual attention through object motion sensitivity [40.966228784674115]
  Active vision enables dynamic visual perception, offering an alternative to static feedforward architectures in computer vision. Event-based cameras, inspired by the mammalian retina, enhance this capability by capturing asynchronous scene changes. To distinguish moving objects while the event-based camera is in motion the agent requires an object motion segmentation mechanism. This work presents a Convolutional Neural Network bio-inspired attention system for selective attention through object motion sensitivity.
  arXiv  Detail & Related papers  (2025-02-10T18:16:30Z)
• Hardware-Algorithm Re-engineering of Retinal Circuit for Intelligent Object Motion Segmentation [0.0]
  We focus on a fundamental visual feature within the mammalian retina, Object Motion Sensitivity (OMS) We present novel CMOS circuits that implement OMS functionality inside image sensors. We verify the functionality and re-configurability of the proposed CMOS circuit designs through Cadence simulations in 180nm technology.
  arXiv  Detail & Related papers  (2024-07-31T20:35:11Z)
• SaccadeDet: A Novel Dual-Stage Architecture for Rapid and Accurate Detection in Gigapixel Images [50.742420049839474]
  'SaccadeDet' is an innovative architecture for gigapixel-level object detection, inspired by the human eye saccadic movement. Our approach, evaluated on the PANDA dataset, achieves an 8x speed increase over the state-of-the-art methods. It also demonstrates significant potential in gigapixel-level pathology analysis through its application to Whole Slide Imaging.
  arXiv  Detail & Related papers  (2024-07-25T11:22:54Z)
• Neuromorphic Synergy for Video Binarization [54.195375576583864]
  Bimodal objects serve as a visual form to embed information that can be easily recognized by vision systems. Neuromorphic cameras offer new capabilities for alleviating motion blur, but it is non-trivial to first de-blur and then binarize the images in a real-time manner. We propose an event-based binary reconstruction method that leverages the prior knowledge of the bimodal target's properties to perform inference independently in both event space and image space. We also develop an efficient integration method to propagate this binary image to high frame rate binary video.
  arXiv  Detail & Related papers  (2024-02-20T01:43:51Z)
• EventTransAct: A video transformer-based framework for Event-camera based action recognition [52.537021302246664]
  Event cameras offer new opportunities compared to standard action recognition in RGB videos. In this study, we employ a computationally efficient model, namely the video transformer network (VTN), which initially acquires spatial embeddings per event-frame. In order to better adopt the VTN for the sparse and fine-grained nature of event data, we design Event-Contrastive Loss ($mathcalL_EC$) and event-specific augmentations.
  arXiv  Detail & Related papers  (2023-08-25T23:51:07Z)
• Object Motion Sensitivity: A Bio-inspired Solution to the Ego-motion Problem for Event-based Cameras [0.0]
  We highlight the capability of the second generation of neuromorphic image sensors, Integrated Retinal Functionality in CMOS Image Sensors (IRIS) IRIS aims to mimic full retinal computations from photoreceptors to output of the retina for targeted feature-extraction. Our results show that OMS can accomplish standard computer vision tasks with similar efficiency to conventional RGB and DVS solutions but offers drastic bandwidth reduction.
  arXiv  Detail & Related papers  (2023-03-24T16:22:06Z)
• Differentiable Frequency-based Disentanglement for Aerial Video Action Recognition [56.91538445510214]
  We present a learning algorithm for human activity recognition in videos. Our approach is designed for UAV videos, which are mainly acquired from obliquely placed dynamic cameras. We conduct extensive experiments on the UAV Human dataset and the NEC Drone dataset.
  arXiv  Detail & Related papers  (2022-09-15T22:16:52Z)
• CNN-based Omnidirectional Object Detection for HermesBot Autonomous Delivery Robot with Preliminary Frame Classification [53.56290185900837]
  We propose an algorithm for optimizing a neural network for object detection using preliminary binary frame classification. An autonomous mobile robot with 6 rolling-shutter cameras on the perimeter providing a 360-degree field of view was used as the experimental setup.
  arXiv  Detail & Related papers  (2021-10-22T15:05:37Z)
• EfficientPhys: Enabling Simple, Fast and Accurate Camera-Based Vitals Measurement [5.435325323159416]
  We propose two novel neural models for camera-based physiological measurement called EfficientPhys. Our models achieve state-of-the-art accuracy on three public datasets.
  arXiv  Detail & Related papers  (2021-10-09T03:51:26Z)
• Deep Homography Estimation in Dynamic Surgical Scenes for Laparoscopic Camera Motion Extraction [6.56651216023737]
  We introduce a method that allows to extract a laparoscope holder's actions from videos of laparoscopic interventions. We synthetically add camera motion to a newly acquired dataset of camera motion free da Vinci surgery image sequences. We find our method transfers from our camera motion free da Vinci surgery dataset to videos of laparoscopic interventions, outperforming classical homography estimation approaches in both, precision by 41%, and runtime on a CPU by 43%.
  arXiv  Detail & Related papers  (2021-09-30T13:05:37Z)
• Optical Flow Estimation from a Single Motion-blurred Image [66.2061278123057]
  Motion blur in an image may have practical interests in fundamental computer vision problems. We propose a novel framework to estimate optical flow from a single motion-blurred image in an end-to-end manner.
  arXiv  Detail & Related papers  (2021-03-04T12:45:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.