Event-Based high-speed low-latency fiducial marker tracking

• URL: http://arxiv.org/abs/2110.05819v1
• Date: Tue, 12 Oct 2021 08:34:31 GMT
• Title: Event-Based high-speed low-latency fiducial marker tracking
• Authors: Adam Loch, Germain Haessig, Markus Vincze
• Abstract summary: We propose an end-to-end pipeline for real-time, low latency, 6 degrees-of-freedom pose estimation of fiducial markers. We employ the high-speed abilities of event-based sensors to directly refine the spatial transformation. This approach allows us to achieve pose estimation at a rate up to 156kHz, while only relying on CPU resources.
• Score: 15.052022635853799
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Motion and dynamic environments, especially under challenging lighting conditions, are still an open issue for robust robotic applications. In this paper, we propose an end-to-end pipeline for real-time, low latency, 6 degrees-of-freedom pose estimation of fiducial markers. Instead of achieving a pose estimation through a conventional frame-based approach, we employ the high-speed abilities of event-based sensors to directly refine the spatial transformation, using consecutive events. Furthermore, we introduce a novel two-way verification process for detecting tracking errors by backtracking the estimated pose, allowing us to evaluate the quality of our tracking. This approach allows us to achieve pose estimation at a rate up to 156~kHz, while only relying on CPU resources. The average end-to-end latency of our method is 3~ms. Experimental results demonstrate outstanding potential for robotic tasks, such as visual servoing in fast action-perception loops.

Related papers

• Event-based vision for egomotion estimation using precise event timing [0.6262316762195913]
  Egomotion estimation is crucial for applications such as autonomous navigation and robotics. Traditional methods relying on inertial sensors are sensitive to external conditions. Vision-based methods provide an efficient alternative by capturing data only when changes are perceived in the scene.
  arXiv  Detail & Related papers  (2025-01-20T15:41:33Z)
• Event-Based Tracking Any Point with Motion-Augmented Temporal Consistency [58.719310295870024]
  This paper presents an event-based framework for tracking any point. It tackles the challenges posed by spatial sparsity and motion sensitivity in events. It achieves 150% faster processing with competitive model parameters.
  arXiv  Detail & Related papers  (2024-12-02T09:13:29Z)
• Exploring Dynamic Transformer for Efficient Object Tracking [58.120191254379854]
  We propose DyTrack, a dynamic transformer framework for efficient tracking. DyTrack automatically learns to configure proper reasoning routes for various inputs, gaining better utilization of the available computational budget. Experiments on multiple benchmarks demonstrate that DyTrack achieves promising speed-precision trade-offs with only a single model.
  arXiv  Detail & Related papers  (2024-03-26T12:31:58Z)
• PNAS-MOT: Multi-Modal Object Tracking with Pareto Neural Architecture Search [64.28335667655129]
  Multiple object tracking is a critical task in autonomous driving. As tracking accuracy improves, neural networks become increasingly complex, posing challenges for their practical application in real driving scenarios due to the high level of latency. In this paper, we explore the use of the neural architecture search (NAS) methods to search for efficient architectures for tracking, aiming for low real-time latency while maintaining relatively high accuracy.
  arXiv  Detail & Related papers  (2024-03-23T04:18:49Z)
• EV-Catcher: High-Speed Object Catching Using Low-latency Event-based Neural Networks [107.62975594230687]
  We demonstrate an application where event cameras excel: accurately estimating the impact location of fast-moving objects. We introduce a lightweight event representation called Binary Event History Image (BEHI) to encode event data at low latency. We show that the system is capable of achieving a success rate of 81% in catching balls targeted at different locations, with a velocity of up to 13 m/s even on compute-constrained embedded platforms.
  arXiv  Detail & Related papers  (2023-04-14T15:23:28Z)
• Fast Event-based Optical Flow Estimation by Triplet Matching [13.298845944779108]
  Event cameras offer advantages over traditional cameras (low latency, high dynamic range, low power, etc.) Optical flow estimation methods that work on packets of events trade off speed for accuracy. We propose a novel optical flow estimation scheme based on triplet matching.
  arXiv  Detail & Related papers  (2022-12-23T09:12:16Z)
• Are We Ready for Vision-Centric Driving Streaming Perception? The ASAP Benchmark [23.872360763782037]
  ASAP is the first benchmark to evaluate the online performance of vision-centric perception in autonomous driving. We propose an annotation-extending pipeline to generate high-frame-rate labels for the 12Hz raw images. In the ASAP benchmark, comprehensive experiment results reveal that the model rank alters under different constraints.
  arXiv  Detail & Related papers  (2022-12-17T16:32:15Z)
• Recurrent Vision Transformers for Object Detection with Event Cameras [62.27246562304705]
  We present Recurrent Vision Transformers (RVTs), a novel backbone for object detection with event cameras. RVTs can be trained from scratch to reach state-of-the-art performance on event-based object detection. Our study brings new insights into effective design choices that can be fruitful for research beyond event-based vision.
  arXiv  Detail & Related papers  (2022-12-11T20:28:59Z)
• PUCK: Parallel Surface and Convolution-kernel Tracking for Event-Based Cameras [4.110120522045467]
  Event-cameras can guarantee fast visual sensing in dynamic environments, but require a tracking algorithm that can keep up with the high data rate induced by the robot ego-motion. We introduce a novel tracking method that leverages the Exponential Reduced Ordinal Surface (EROS) data representation to decouple event-by-event processing and tracking. We propose the task of tracking the air hockey puck sliding on a surface, with the future aim of controlling the iCub robot to reach the target precisely and on time.
  arXiv  Detail & Related papers  (2022-05-16T13:23:52Z)
• Predictive Visual Tracking: A New Benchmark and Baseline Approach [27.87099869398515]
  In the real-world scenarios, the onboard processing time of the image streams inevitably leads to a discrepancy between the tracking results and the real-world states. Existing visual tracking benchmarks commonly run the trackers offline and ignore such latency in the evaluation. In this work, we aim to deal with a more realistic problem of latency-aware tracking.
  arXiv  Detail & Related papers  (2021-03-08T01:50:05Z)
• Towards Streaming Perception [70.68520310095155]
  We present an approach that coherently integrates latency and accuracy into a single metric for real-time online perception. The key insight behind this metric is to jointly evaluate the output of the entire perception stack at every time instant. We focus on the illustrative tasks of object detection and instance segmentation in urban video streams, and contribute a novel dataset with high-quality and temporally-dense annotations.
  arXiv  Detail & Related papers  (2020-05-21T01:51:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.