How Many Events do You Need? Event-based Visual Place Recognition Using Sparse But Varying Pixels

• URL: http://arxiv.org/abs/2206.13673v1
• Date: Tue, 28 Jun 2022 00:24:12 GMT
• Title: How Many Events do You Need? Event-based Visual Place Recognition Using Sparse But Varying Pixels
• Authors: Tobias Fischer and Michael Milford
• Abstract summary: One of the potential applications of event camera research lies in visual place recognition for robot localization. We show that the absolute difference in the number of events at those pixel locations accumulated into event frames can be sufficient for the place recognition task. We evaluate our proposed approach on the Brisbane-Event-VPR dataset in an outdoor driving scenario, as well as the newly contributed indoor QCR-Event-VPR dataset.
• Score: 29.6328152991222
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Event cameras continue to attract interest due to desirable characteristics such as high dynamic range, low latency, virtually no motion blur, and high energy efficiency. One of the potential applications of event camera research lies in visual place recognition for robot localization, where a query observation has to be matched to the corresponding reference place in the database. In this letter, we explore the distinctiveness of event streams from a small subset of pixels (in the tens or hundreds). We demonstrate that the absolute difference in the number of events at those pixel locations accumulated into event frames can be sufficient for the place recognition task, when pixels that display large variations in the reference set are used. Using such sparse (over image coordinates) but varying (variance over the number of events per pixel location) pixels enables frequent and computationally cheap updates of the location estimates. Furthermore, when event frames contain a constant number of events, our method takes full advantage of the event-driven nature of the sensory stream and displays promising robustness to changes in velocity. We evaluate our proposed approach on the Brisbane-Event-VPR dataset in an outdoor driving scenario, as well as the newly contributed indoor QCR-Event-VPR dataset that was captured with a DAVIS346 camera mounted on a mobile robotic platform. Our results show that our approach achieves competitive performance when compared to several baseline methods on those datasets, and is particularly well suited for compute- and energy-constrained platforms such as interplanetary rovers.

Related papers

• Event-to-Video Conversion for Overhead Object Detection [7.744259147081667]
  Event cameras complicate downstream image processing, especially for complex tasks such as object detection. We show that there is a significant gap in performance between dense event representations and corresponding RGB frames. We apply event-to-video conversion models that convert event streams into gray-scale video to close this gap.
  arXiv  Detail & Related papers  (2024-02-09T22:07:39Z)
• Rethinking Event-based Human Pose Estimation with 3D Event Representations [26.592295349210787]
  Event cameras offer a robust solution for navigating challenging contexts. We introduce two 3D event representations: the Rasterized Event Point Cloud and the Decoupled Event Voxel. Experiments on EV-3DPW demonstrate that the robustness of our proposed 3D representation methods compared to traditional RGB images and event frame techniques.
  arXiv  Detail & Related papers  (2023-11-08T10:45:09Z)
• EvDNeRF: Reconstructing Event Data with Dynamic Neural Radiance Fields [80.94515892378053]
  EvDNeRF is a pipeline for generating event data and training an event-based dynamic NeRF. NeRFs offer geometric-based learnable rendering, but prior work with events has only considered reconstruction of static scenes. We show that by training on varied batch sizes of events, we can improve test-time predictions of events at fine time resolutions.
  arXiv  Detail & Related papers  (2023-10-03T21:08:41Z)
• EventTransAct: A video transformer-based framework for Event-camera based action recognition [52.537021302246664]
  Event cameras offer new opportunities compared to standard action recognition in RGB videos. In this study, we employ a computationally efficient model, namely the video transformer network (VTN), which initially acquires spatial embeddings per event-frame. In order to better adopt the VTN for the sparse and fine-grained nature of event data, we design Event-Contrastive Loss ($mathcalL_EC$) and event-specific augmentations.
  arXiv  Detail & Related papers  (2023-08-25T23:51:07Z)
• On the Generation of a Synthetic Event-Based Vision Dataset for Navigation and Landing [69.34740063574921]
  This paper presents a methodology for generating event-based vision datasets from optimal landing trajectories. We construct sequences of photorealistic images of the lunar surface with the Planet and Asteroid Natural Scene Generation Utility. We demonstrate that the pipeline can generate realistic event-based representations of surface features by constructing a dataset of 500 trajectories.
  arXiv  Detail & Related papers  (2023-08-01T09:14:20Z)
• Cross-modal Place Recognition in Image Databases using Event-based Sensors [28.124708490967713]
  We present the first cross-modal visual place recognition framework that is capable of retrieving regular images from a database given an event query. Our method demonstrates promising results with respect to the state-of-the-art frame-based and event-based methods on the Brisbane-Event-VPR dataset.
  arXiv  Detail & Related papers  (2023-07-03T14:24:04Z)
• EV-Catcher: High-Speed Object Catching Using Low-latency Event-based Neural Networks [107.62975594230687]
  We demonstrate an application where event cameras excel: accurately estimating the impact location of fast-moving objects. We introduce a lightweight event representation called Binary Event History Image (BEHI) to encode event data at low latency. We show that the system is capable of achieving a success rate of 81% in catching balls targeted at different locations, with a velocity of up to 13 m/s even on compute-constrained embedded platforms.
  arXiv  Detail & Related papers  (2023-04-14T15:23:28Z)
• Dual Memory Aggregation Network for Event-Based Object Detection with Learnable Representation [79.02808071245634]
  Event-based cameras are bio-inspired sensors that capture brightness change of every pixel in an asynchronous manner. Event streams are divided into grids in the x-y-t coordinates for both positive and negative polarity, producing a set of pillars as 3D tensor representation. Long memory is encoded in the hidden state of adaptive convLSTMs while short memory is modeled by computing spatial-temporal correlation between event pillars.
  arXiv  Detail & Related papers  (2023-03-17T12:12:41Z)
• Bridging the Gap between Events and Frames through Unsupervised Domain Adaptation [57.22705137545853]
  We propose a task transfer method that allows models to be trained directly with labeled images and unlabeled event data. We leverage the generative event model to split event features into content and motion features. Our approach unlocks the vast amount of existing image datasets for the training of event-based neural networks.
  arXiv  Detail & Related papers  (2021-09-06T17:31:37Z)
• Learning to Detect Objects with a 1 Megapixel Event Camera [14.949946376335305]
  Event cameras encode visual information with high temporal precision, low data-rate, and high-dynamic range. Due to the novelty of the field, the performance of event-based systems on many vision tasks is still lower compared to conventional frame-based solutions.
  arXiv  Detail & Related papers  (2020-09-28T16:03:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.