1000 Rallies: An Event-Camera Dataset and Real-Time Learned Ball-State Estimation for Robotic Table Tennis

• URL: http://arxiv.org/abs/2606.25620v1
• Date: Wed, 24 Jun 2026 09:28:21 GMT
• Title: 1000 Rallies: An Event-Camera Dataset and Real-Time Learned Ball-State Estimation for Robotic Table Tennis
• Abstract summary: We introduce the first large-scale event-camera dataset for table tennis, comprising over 1000 rallies from a diverse group of players.<n>We train a convolutional neural network robust to background player motion that jointly estimates the ball's position and velocity in the image-plane from events.<n>We close the perception-action loop by integrating the event-based system with a Stubli robotic arm.
• Score: 9.183928190216532
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Robotic table tennis has emerged as a compelling benchmark for real-time robotic perception due to its fast ball dynamics and stringent timing requirements. Accurate, high-frequency, and low-latency ball state estimation is critical for reliable trajectory prediction and timely control. Traditional frame-based cameras face an inherent trade-off: low frame rates leave temporal blind spots that miss fast-moving objects and high frame rates raise data and computational cost. Event cameras instead offer microsecond temporal resolution and, under sufficient illumination, remain largely free of motion blur even at high ball speeds. However, the community lacks large-scale datasets to develop and benchmark event-based perception in realistic sports scenarios. We address this gap by introducing the first large-scale event-camera dataset for table tennis, comprising over 1000 rallies from a diverse group of players ranging from amateurs to elite-level athletes. Each recording captures the event stream alongside 14 synchronized high-speed frame-based cameras at 200 FPS, which we use to produce 1 kHz pseudo ground-truth labels for ball position, velocity, and spin. Building on this dataset, we train a convolutional neural network robust to background player motion that jointly estimates the ball's position and velocity in the image-plane from events. Treating the predicted velocity as an additional measurement in the Kalman filter reduces bounce-point prediction error by 36% relative to a position-only baseline. Finally, we close the perception-action loop by integrating the event-based system with a Stäubli robotic arm, enabling the first real-time human-robot table tennis rallies driven by event-based perception.

Related papers

• Event-based Gaze Control System for Accurate Real-time Spin Estimation in Professional Ball Games [11.203086649949185]
  We propose an event-based active vision system that can track balls and measure their spin in real time.<n>The system consists of an event camera for its high temporal resolution and minimal motion blur, high-speed pan/tilt galvanometer mirrors to keep the ball in the field of view, and a low-tunable lens to increase the spatial resolution on the ball and keep it in focus.
  arXiv  Detail & Related papers  (2026-06-25T09:14:03Z)
• Event-Based Visual Teach-and-Repeat via Fast Fourier-Domain Cross-Correlation [52.46888249268445]
  We present the first event-camera-based visual teach-and-repeat system.<n>We develop a frequency-domain cross-correlation framework that transforms the event stream matching problem into computationally efficient space multiplications.<n>Experiments using a Prophesee EVK4 HD event camera mounted on an AgileX Scout Mini robot demonstrate successful autonomous navigation.
  arXiv  Detail & Related papers  (2025-09-21T23:53:31Z)
• Egocentric Event-Based Vision for Ping Pong Ball Trajectory Prediction [17.147140984254655]
  We present a real-time egocentric trajectory prediction system for table tennis using event cameras.<n>We collect a dataset of ping-pong game sequences, including 3D ground-truth trajectories of the ball, synchronized with sensor data from the Meta Project Aria glasses.<n>Our detection pipeline has a worst-case total latency of 4.5 ms, including computation and perception.
  arXiv  Detail & Related papers  (2025-06-09T15:22:55Z)
• An Event-Based Perception Pipeline for a Table Tennis Robot [12.101426862186072]
  We present the first real-time perception pipeline for a table tennis robot that uses only event-based cameras.<n>We show that compared to a frame-based pipeline, event-based perception pipelines have an update rate which is an order of magnitude higher.
  arXiv  Detail & Related papers  (2025-02-02T10:56:37Z)
• Deblur e-NeRF: NeRF from Motion-Blurred Events under High-speed or Low-light Conditions [56.84882059011291]
  We propose Deblur e-NeRF, a novel method to reconstruct blur-minimal NeRFs from motion-red events. We also introduce a novel threshold-normalized total variation loss to improve the regularization of large textureless patches.
  arXiv  Detail & Related papers  (2024-09-26T15:57:20Z)
• Table tennis ball spin estimation with an event camera [11.735290341808064]
  In table tennis, the combination of high velocity and spin renders traditional low frame rate cameras inadequate. We present the first method for table tennis spin estimation using an event camera. We achieve a spin magnitude mean error of $10.7 pm 17.3$ rps and a spin axis mean error of $32.9 pm 38.2deg$ in real time for a flying ball.
  arXiv  Detail & Related papers  (2024-04-15T15:36:38Z)
• EvDNeRF: Reconstructing Event Data with Dynamic Neural Radiance Fields [80.94515892378053]
  EvDNeRF is a pipeline for generating event data and training an event-based dynamic NeRF. NeRFs offer geometric-based learnable rendering, but prior work with events has only considered reconstruction of static scenes. We show that by training on varied batch sizes of events, we can improve test-time predictions of events at fine time resolutions.
  arXiv  Detail & Related papers  (2023-10-03T21:08:41Z)
• EV-Catcher: High-Speed Object Catching Using Low-latency Event-based Neural Networks [107.62975594230687]
  We demonstrate an application where event cameras excel: accurately estimating the impact location of fast-moving objects. We introduce a lightweight event representation called Binary Event History Image (BEHI) to encode event data at low latency. We show that the system is capable of achieving a success rate of 81% in catching balls targeted at different locations, with a velocity of up to 13 m/s even on compute-constrained embedded platforms.
  arXiv  Detail & Related papers  (2023-04-14T15:23:28Z)
• Fast Trajectory End-Point Prediction with Event Cameras for Reactive Robot Control [4.110120522045467]
  In this paper, we propose to exploit the low latency, motion-driven sampling, and data compression properties of event cameras to overcome these issues. As a use-case, we use a Panda robotic arm to intercept a ball bouncing on a table. We train the network in simulation to speed up the dataset acquisition and then fine-tune the models on real trajectories.
  arXiv  Detail & Related papers  (2023-02-27T14:14:52Z)
• ESL: Event-based Structured Light [62.77144631509817]
  Event cameras are bio-inspired sensors providing significant advantages over standard cameras. We propose a novel structured-light system using an event camera to tackle the problem of accurate and high-speed depth sensing.
  arXiv  Detail & Related papers  (2021-11-30T15:47:39Z)
• EventHands: Real-Time Neural 3D Hand Reconstruction from an Event Stream [80.15360180192175]
  3D hand pose estimation from monocular videos is a long-standing and challenging problem. We address it for the first time using a single event camera, i.e., an asynchronous vision sensor reacting on brightness changes. Our approach has characteristics previously not demonstrated with a single RGB or depth camera.
  arXiv  Detail & Related papers  (2020-12-11T16:45:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.