Multi-Camera Asynchronous Ball Localization and Trajectory Prediction with Factor Graphs and Human Poses

• URL: http://arxiv.org/abs/2401.17185v1
• Date: Tue, 30 Jan 2024 17:13:29 GMT
• Title: Multi-Camera Asynchronous Ball Localization and Trajectory Prediction with Factor Graphs and Human Poses
• Authors: Qingyu Xiao, Zulfiqar Zaidi and Matthew Gombolay
• Abstract summary: The rapid and precise localization and prediction of a ball are critical for developing agile robots in ball sports. We introduce an innovative approach that combines a multi-camera system with factor graphs for real-time and asynchronous 3D tennis ball localization. To enhance spin inference early in the ball's flight, where limited observations are available, we integrate human pose data using a temporal convolutional network (TCN) Our result shows the trained TCN can predict the spin priors with RMSE of 5.27 Hz.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The rapid and precise localization and prediction of a ball are critical for developing agile robots in ball sports, particularly in sports like tennis characterized by high-speed ball movements and powerful spins. The Magnus effect induced by spin adds complexity to trajectory prediction during flight and bounce dynamics upon contact with the ground. In this study, we introduce an innovative approach that combines a multi-camera system with factor graphs for real-time and asynchronous 3D tennis ball localization. Additionally, we estimate hidden states like velocity and spin for trajectory prediction. Furthermore, to enhance spin inference early in the ball's flight, where limited observations are available, we integrate human pose data using a temporal convolutional network (TCN) to compute spin priors within the factor graph. This refinement provides more accurate spin priors at the beginning of the factor graph, leading to improved early-stage hidden state inference for prediction. Our result shows the trained TCN can predict the spin priors with RMSE of 5.27 Hz. Integrating TCN into the factor graph reduces the prediction error of landing positions by over 63.6% compared to a baseline method that utilized an adaptive extended Kalman filter.

Related papers

• Adapting to Length Shift: FlexiLength Network for Trajectory Prediction [53.637837706712794]
  Trajectory prediction plays an important role in various applications, including autonomous driving, robotics, and scene understanding. Existing approaches mainly focus on developing compact neural networks to increase prediction precision on public datasets, typically employing a standardized input duration. We introduce a general and effective framework, the FlexiLength Network (FLN), to enhance the robustness of existing trajectory prediction against varying observation periods.
  arXiv  Detail & Related papers  (2024-03-31T17:18:57Z)
• TrajectoryNAS: A Neural Architecture Search for Trajectory Prediction [0.0]
  Trajectory prediction is a critical component of autonomous driving systems. This paper introduces TrajectoryNAS, a pioneering method that focuses on utilizing point cloud data for trajectory prediction.
  arXiv  Detail & Related papers  (2024-03-18T11:48:41Z)
• Context-based Interpretable Spatio-Temporal Graph Convolutional Network for Human Motion Forecasting [0.0]
  We present a Context- Interpretable Stemporal Graphal Network (IST-GCN) as an efficient 3D human pose forecasting model. Our architecture extracts meaningful information from pose sequences, aggregates displacements and accelerations into the input model, and finally predicts the output displacements.
  arXiv  Detail & Related papers  (2024-02-21T17:51:30Z)
• Black-Box vs. Gray-Box: A Case Study on Learning Table Tennis Ball Trajectory Prediction with Spin and Impacts [12.103456313315764]
  We present a method for table tennis ball trajectory filtering and prediction. We use data to learn parameters of the dynamics model, of an extended Kalman filter, and of a neural model that infers the ball's initial condition.
  arXiv  Detail & Related papers  (2023-05-24T14:28:22Z)
• PREF: Predictability Regularized Neural Motion Fields [68.60019434498703]
  Knowing 3D motions in a dynamic scene is essential to many vision applications. We leverage a neural motion field for estimating the motion of all points in a multiview setting. We propose to regularize the estimated motion to be predictable.
  arXiv  Detail & Related papers  (2022-09-21T22:32:37Z)
• DMS-GCN: Dynamic Mutiscale Spatiotemporal Graph Convolutional Networks for Human Motion Prediction [8.142947808507365]
  We propose a feed-forward deep neural network for motion prediction. The entire model is suitable for all actions and follows a framework of encoder-decoder. Our approach outperforms SOTA methods on the datasets of Human3.6M and CMU Mocap.
  arXiv  Detail & Related papers  (2021-12-20T07:07:03Z)
• Multiscale Spatio-Temporal Graph Neural Networks for 3D Skeleton-Based Motion Prediction [92.16318571149553]
  We propose a multiscale-temporal graph neural network (MST-GNN) to predict the future 3D-based skeleton human poses. The MST-GNN outperforms state-of-the-art methods in both short and long-term motion prediction.
  arXiv  Detail & Related papers  (2021-08-25T14:05:37Z)
• SGCN:Sparse Graph Convolution Network for Pedestrian Trajectory Prediction [64.16212996247943]
  We present a Sparse Graph Convolution Network(SGCN) for pedestrian trajectory prediction. Specifically, the SGCN explicitly models the sparse directed interaction with a sparse directed spatial graph to capture adaptive interaction pedestrians. visualizations indicate that our method can capture adaptive interactions between pedestrians and their effective motion tendencies.
  arXiv  Detail & Related papers  (2021-04-04T03:17:42Z)
• AVGCN: Trajectory Prediction using Graph Convolutional Networks Guided by Human Attention [11.342351420439725]
  We propose a novel method, AVGCN, for trajectory prediction utilizing graph convolutional networks (GCN) based on human attention. Our approach achieves state-of-the-art performance on several trajectory prediction benchmarks, and the lowest average prediction error over all considered benchmarks.
  arXiv  Detail & Related papers  (2021-01-14T16:00:31Z)
• Spatio-Temporal Graph Scattering Transform [54.52797775999124]
  Graph neural networks may be impractical in some real-world scenarios due to a lack of sufficient high-quality training data. We put forth a novel mathematically designed framework to analyze-temporal data.
  arXiv  Detail & Related papers  (2020-12-06T19:49:55Z)
• Adversarial Refinement Network for Human Motion Prediction [61.50462663314644]
  Two popular methods, recurrent neural networks and feed-forward deep networks, are able to predict rough motion trend. We propose an Adversarial Refinement Network (ARNet) following a simple yet effective coarse-to-fine mechanism with novel adversarial error augmentation.
  arXiv  Detail & Related papers  (2020-11-23T05:42:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.