The Spatio-Temporal Poisson Point Process: A Simple Model for the Alignment of Event Camera Data

• URL: http://arxiv.org/abs/2106.06887v1
• Date: Sun, 13 Jun 2021 00:43:27 GMT
• Title: The Spatio-Temporal Poisson Point Process: A Simple Model for the Alignment of Event Camera Data
• Authors: Cheng Gu, Erik Learned-Miller, Daniel Sheldon, Guillermo Gallego, Pia Bideau
• Abstract summary: Event cameras provide a natural and data efficient representation of visual information. We propose a new model of event data that captures its natural-temporal structure. We show new state of the art accuracy for rotational velocity estimation on the DAVIS 240C dataset.
• Score: 19.73526916714181
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Event cameras, inspired by biological vision systems, provide a natural and data efficient representation of visual information. Visual information is acquired in the form of events that are triggered by local brightness changes. Each pixel location of the camera's sensor records events asynchronously and independently with very high temporal resolution. However, because most brightness changes are triggered by relative motion of the camera and the scene, the events recorded at a single sensor location seldom correspond to the same world point. To extract meaningful information from event cameras, it is helpful to register events that were triggered by the same underlying world point. In this work we propose a new model of event data that captures its natural spatio-temporal structure. We start by developing a model for aligned event data. That is, we develop a model for the data as though it has been perfectly registered already. In particular, we model the aligned data as a spatio-temporal Poisson point process. Based on this model, we develop a maximum likelihood approach to registering events that are not yet aligned. That is, we find transformations of the observed events that make them as likely as possible under our model. In particular we extract the camera rotation that leads to the best event alignment. We show new state of the art accuracy for rotational velocity estimation on the DAVIS 240C dataset. In addition, our method is also faster and has lower computational complexity than several competing methods.

Related papers

• EF-3DGS: Event-Aided Free-Trajectory 3D Gaussian Splatting [76.02450110026747]
  Event cameras, inspired by biological vision, record pixel-wise intensity changes asynchronously with high temporal resolution. We propose Event-Aided Free-Trajectory 3DGS, which seamlessly integrates the advantages of event cameras into 3DGS. We evaluate our method on the public Tanks and Temples benchmark and a newly collected real-world dataset, RealEv-DAVIS.
  arXiv  Detail & Related papers  (2024-10-20T13:44:24Z)
• Evaluating Image-Based Face and Eye Tracking with Event Cameras [9.677797822200965]
  Event Cameras, also known as Neuromorphic sensors, capture changes in local light intensity at the pixel level, producing asynchronously generated data termed events'' This data format mitigates common issues observed in conventional cameras, like under-sampling when capturing fast-moving objects. We evaluate the viability of integrating conventional algorithms with event-based data, transformed into a frame format.
  arXiv  Detail & Related papers  (2024-08-19T20:27:08Z)
• Deformable Neural Radiance Fields using RGB and Event Cameras [65.40527279809474]
  We develop a novel method to model the deformable neural radiance fields using RGB and event cameras. The proposed method uses the asynchronous stream of events and sparse RGB frames. Experiments conducted on both realistically rendered graphics and real-world datasets demonstrate a significant benefit of the proposed method.
  arXiv  Detail & Related papers  (2023-09-15T14:19:36Z)
• A Temporal Densely Connected Recurrent Network for Event-based Human Pose Estimation [24.367222637492787]
  Event camera is an emerging bio-inspired vision sensors that report per-pixel brightness changes asynchronously. This paper proposes a novel densely connected recurrent architecture to address the problem of incomplete information. By this recurrent architecture, we can explicitly model not only the sequential but also non-sequential geometric consistency across time steps.
  arXiv  Detail & Related papers  (2022-09-15T04:08:18Z)
• How Many Events do You Need? Event-based Visual Place Recognition Using Sparse But Varying Pixels [29.6328152991222]
  One of the potential applications of event camera research lies in visual place recognition for robot localization. We show that the absolute difference in the number of events at those pixel locations accumulated into event frames can be sufficient for the place recognition task. We evaluate our proposed approach on the Brisbane-Event-VPR dataset in an outdoor driving scenario, as well as the newly contributed indoor QCR-Event-VPR dataset.
  arXiv  Detail & Related papers  (2022-06-28T00:24:12Z)
• E$^2$(GO)MOTION: Motion Augmented Event Stream for Egocentric Action Recognition [21.199869051111367]
  Event cameras capture pixel-level intensity changes in the form of "events" N-EPIC-Kitchens is the first event-based camera extension of the large-scale EPIC-Kitchens dataset. We show that event data provides a comparable performance to RGB and optical flow, yet without any additional flow computation at deploy time.
  arXiv  Detail & Related papers  (2021-12-07T09:43:08Z)
• Bridging the Gap between Events and Frames through Unsupervised Domain Adaptation [57.22705137545853]
  We propose a task transfer method that allows models to be trained directly with labeled images and unlabeled event data. We leverage the generative event model to split event features into content and motion features. Our approach unlocks the vast amount of existing image datasets for the training of event-based neural networks.
  arXiv  Detail & Related papers  (2021-09-06T17:31:37Z)
• VisEvent: Reliable Object Tracking via Collaboration of Frame and Event Flows [93.54888104118822]
  We propose a large-scale Visible-Event benchmark (termed VisEvent) due to the lack of a realistic and scaled dataset for this task. Our dataset consists of 820 video pairs captured under low illumination, high speed, and background clutter scenarios. Based on VisEvent, we transform the event flows into event images and construct more than 30 baseline methods.
  arXiv  Detail & Related papers  (2021-08-11T03:55:12Z)
• EventHands: Real-Time Neural 3D Hand Reconstruction from an Event Stream [80.15360180192175]
  3D hand pose estimation from monocular videos is a long-standing and challenging problem. We address it for the first time using a single event camera, i.e., an asynchronous vision sensor reacting on brightness changes. Our approach has characteristics previously not demonstrated with a single RGB or depth camera.
  arXiv  Detail & Related papers  (2020-12-11T16:45:34Z)
• Unsupervised Feature Learning for Event Data: Direct vs Inverse Problem Formulation [53.850686395708905]
  Event-based cameras record an asynchronous stream of per-pixel brightness changes. In this paper, we focus on single-layer architectures for representation learning from event data. We show improvements of up to 9 % in the recognition accuracy compared to the state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-09-23T10:40:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.