6-DoF Object Tracking with Event-based Optical Flow and Frames

• URL: http://arxiv.org/abs/2508.14776v1
• Date: Wed, 20 Aug 2025 15:22:51 GMT
• Title: 6-DoF Object Tracking with Event-based Optical Flow and Frames
• Authors: Zhichao Li, Arren Glover, Chiara Bartolozzi, Lorenzo Natale,
• Abstract summary: We propose an event-based optical flow algorithm for object motion measurement to implement an object 6-DoF velocity tracker.<n>By integrating the tracked object 6-DoF velocity with low frequency estimated pose from the global pose estimator, the method can track pose when objects move at high-speed.
• Score: 12.63903994540524
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Tracking the position and orientation of objects in space (i.e., in 6-DoF) in real time is a fundamental problem in robotics for environment interaction. It becomes more challenging when objects move at high-speed due to frame rate limitations in conventional cameras and motion blur. Event cameras are characterized by high temporal resolution, low latency and high dynamic range, that can potentially overcome the impacts of motion blur. Traditional RGB cameras provide rich visual information that is more suitable for the challenging task of single-shot object pose estimation. In this work, we propose using event-based optical flow combined with an RGB based global object pose estimator for 6-DoF pose tracking of objects at high-speed, exploiting the core advantages of both types of vision sensors. Specifically, we propose an event-based optical flow algorithm for object motion measurement to implement an object 6-DoF velocity tracker. By integrating the tracked object 6-DoF velocity with low frequency estimated pose from the global pose estimator, the method can track pose when objects move at high-speed. The proposed algorithm is tested and validated on both synthetic and real world data, demonstrating its effectiveness, especially in high-speed motion scenarios.

Related papers

• PoseStreamer: A Multi-modal Framework for 3D Tracking of Unseen Moving Objects [4.1334804706669095]
  PoseStreamer is a robust multi-modal 6DoF pose estimation framework for high-speed moving scenarios.<n>MoCapCube6D is a novel multi-modal dataset constructed to benchmark performance under rapid motion.
  arXiv  Detail & Related papers  (2025-12-28T15:52:58Z)
• Optical Flow-Guided 6DoF Object Pose Tracking with an Event Camera [18.13747114612191]
  We present an optical flow-guided 6DoF object pose tracking method with an event camera.<n>We show that our methods outperform event-based state-of-the-art methods in terms of both accuracy and robustness.
  arXiv  Detail & Related papers  (2025-12-24T08:40:57Z)
• TOFFE -- Temporally-binned Object Flow from Events for High-speed and Energy-Efficient Object Detection and Tracking [10.458676835674847]
  Event-based cameras offer a biologically-inspired solution to this by capturing only changes in intensity levels at exceptionally high temporal resolution and low power consumption.<n>We propose TOFFE, a lightweight hybrid framework for performing event-based object motion estimation.
  arXiv  Detail & Related papers  (2025-01-21T20:20:34Z)
• MATE: Motion-Augmented Temporal Consistency for Event-based Point Tracking [58.719310295870024]
  This paper presents an event-based framework for tracking any point.<n>To resolve ambiguities caused by event sparsity, a motion-guidance module incorporates kinematic vectors into the local matching process.<n>The method improves the $Survival_50$ metric by 17.9% over event-only tracking of any point baseline.
  arXiv  Detail & Related papers  (2024-12-02T09:13:29Z)
• SpikeMOT: Event-based Multi-Object Tracking with Sparse Motion Features [52.213656737672935]
  SpikeMOT is an event-based multi-object tracker. SpikeMOT uses spiking neural networks to extract sparsetemporal features from event streams associated with objects.
  arXiv  Detail & Related papers  (2023-09-29T05:13:43Z)
• FEDORA: Flying Event Dataset fOr Reactive behAvior [9.470870778715689]
  Event-based sensors have emerged as low latency and low energy alternatives to standard frame-based cameras for capturing high-speed motion.<n>We present Flying Event dataset fOr Reactive behAviour (FEDORA) - a fully synthetic dataset for perception tasks.
  arXiv  Detail & Related papers  (2023-05-22T22:59:05Z)
• Event-based Simultaneous Localization and Mapping: A Comprehensive Survey [52.73728442921428]
  Review of event-based vSLAM algorithms that exploit the benefits of asynchronous and irregular event streams for localization and mapping tasks. Paper categorizes event-based vSLAM methods into four main categories: feature-based, direct, motion-compensation, and deep learning methods.
  arXiv  Detail & Related papers  (2023-04-19T16:21:14Z)
• ParticleSfM: Exploiting Dense Point Trajectories for Localizing Moving Cameras in the Wild [57.37891682117178]
  We present a robust dense indirect structure-from-motion method for videos that is based on dense correspondence from pairwise optical flow. A novel neural network architecture is proposed for processing irregular point trajectory data. Experiments on MPI Sintel dataset show that our system produces significantly more accurate camera trajectories.
  arXiv  Detail & Related papers  (2022-07-19T09:19:45Z)
• ROFT: Real-Time Optical Flow-Aided 6D Object Pose and Velocity Tracking [7.617467911329272]
  We introduce ROFT, a Kalman filtering approach for 6D object pose and velocity tracking from a stream of RGB-D images. By leveraging real-time optical flow, ROFT synchronizes delayed outputs of low frame rate Convolutional Neural Networks for instance segmentation and 6D object pose estimation. Results demonstrate that our approach outperforms state-of-the-art methods for 6D object pose tracking, while also providing 6D object velocity tracking.
  arXiv  Detail & Related papers  (2021-11-06T07:30:00Z)
• Tracking 6-DoF Object Motion from Events and Frames [0.0]
  We propose a novel approach for 6 degree-of-freedom (6-DoF)object motion tracking that combines measurements of eventand frame-based cameras.
  arXiv  Detail & Related papers  (2021-03-29T12:39:38Z)
• Event-based Motion Segmentation with Spatio-Temporal Graph Cuts [51.17064599766138]
  We have developed a method to identify independently objects acquired with an event-based camera. The method performs on par or better than the state of the art without having to predetermine the number of expected moving objects.
  arXiv  Detail & Related papers  (2020-12-16T04:06:02Z)
• Exposure Trajectory Recovery from Motion Blur [90.75092808213371]
  Motion blur in dynamic scenes is an important yet challenging research topic. In this paper, we define exposure trajectories, which represent the motion information contained in a blurry image. A novel motion offset estimation framework is proposed to model pixel-wise displacements of the latent sharp image.
  arXiv  Detail & Related papers  (2020-10-06T05:23:33Z)
• End-to-end Learning of Object Motion Estimation from Retinal Events for Event-based Object Tracking [35.95703377642108]
  We propose a novel deep neural network to learn and regress a parametric object-level motion/transform model for event-based object tracking. To achieve this goal, we propose a synchronous Time-Surface with Linear Time Decay representation. We feed the sequence of TSLTD frames to a novel Retinal Motion Regression Network (RMRNet) perform to an end-to-end 5-DoF object motion regression.
  arXiv  Detail & Related papers  (2020-02-14T08:19:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.