Decoupling Video and Human Motion: Towards Practical Event Detection in Athlete Recordings

• URL: http://arxiv.org/abs/2004.09776v2
• Date: Wed, 22 Apr 2020 15:52:25 GMT
• Title: Decoupling Video and Human Motion: Towards Practical Event Detection in Athlete Recordings
• Authors: Moritz Einfalt, Rainer Lienhart
• Abstract summary: We propose to use 2D human pose sequences as an intermediate representation that decouples human motion from the raw video information. We describe two approaches to event detection on pose sequences and evaluate them in complementary domains: swimming and athletics. Our approach is not limited to these domains and shows the flexibility of pose-based motion event detection.
• Score: 33.770877823910176
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper we address the problem of motion event detection in athlete recordings from individual sports. In contrast to recent end-to-end approaches, we propose to use 2D human pose sequences as an intermediate representation that decouples human motion from the raw video information. Combined with domain-adapted athlete tracking, we describe two approaches to event detection on pose sequences and evaluate them in complementary domains: swimming and athletics. For swimming, we show how robust decision rules on pose statistics can detect different motion events during swim starts, with a F1 score of over 91% despite limited data. For athletics, we use a convolutional sequence model to infer stride-related events in long and triple jump recordings, leading to highly accurate detections with 96% in F1 score at only +/- 5ms temporal deviation. Our approach is not limited to these domains and shows the flexibility of pose-based motion event detection.

Related papers

• SportsSloMo: A New Benchmark and Baselines for Human-centric Video Frame Interpolation [11.198172694893927]
  SportsSloMo is a benchmark consisting of more than 130K video clips and 1M video frames of high-resolution ($geq$720p) slow-motion sports videos crawled from YouTube. We re-train several state-of-the-art methods on our benchmark, and the results show a decrease in their accuracy compared to other datasets. We introduce two loss terms considering the human-aware priors, where we add auxiliary supervision to panoptic segmentation and human keypoints detection.
  arXiv  Detail & Related papers  (2023-08-31T17:23:50Z)
• Bridging the Gap between Events and Frames through Unsupervised Domain Adaptation [57.22705137545853]
  We propose a task transfer method that allows models to be trained directly with labeled images and unlabeled event data. We leverage the generative event model to split event features into content and motion features. Our approach unlocks the vast amount of existing image datasets for the training of event-based neural networks.
  arXiv  Detail & Related papers  (2021-09-06T17:31:37Z)
• EventHPE: Event-based 3D Human Pose and Shape Estimation [33.197194879047956]
  Event camera is an emerging imaging sensor for capturing dynamics of moving objects as events. We propose a two-stage deep learning approach, called EventHPE. The first-stage, FlowNet, is trained by unsupervised learning to infer optical flow from events. The second-stage, ShapeNet, is fed as input to the ShapeNet in the second stage to estimate 3D human shapes.
  arXiv  Detail & Related papers  (2021-08-15T21:40:19Z)
• Multi-level Motion Attention for Human Motion Prediction [132.29963836262394]
  We study the use of different types of attention, computed at joint, body part, and full pose levels. Our experiments on Human3.6M, AMASS and 3DPW validate the benefits of our approach for both periodical and non-periodical actions.
  arXiv  Detail & Related papers  (2021-06-17T08:08:11Z)
• History Repeats Itself: Human Motion Prediction via Motion Attention [81.94175022575966]
  We introduce an attention-based feed-forward network that explicitly leverages the observation that human motion tends to repeat itself. In particular, we propose to extract motion attention to capture the similarity between the current motion context and the historical motion sub-sequences. Our experiments on Human3.6M, AMASS and 3DPW evidence the benefits of our approach for both periodical and non-periodical actions.
  arXiv  Detail & Related papers  (2020-07-23T02:12:27Z)
• Contact and Human Dynamics from Monocular Video [73.47466545178396]
  Existing deep models predict 2D and 3D kinematic poses from video that are approximately accurate, but contain visible errors. We present a physics-based method for inferring 3D human motion from video sequences that takes initial 2D and 3D pose estimates as input.
  arXiv  Detail & Related papers  (2020-07-22T21:09:11Z)
• Fusing Motion Patterns and Key Visual Information for Semantic Event Recognition in Basketball Videos [87.29451470527353]
  We propose a scheme to fuse global and local motion patterns (MPs) and key visual information (KVI) for semantic event recognition in basketball videos. An algorithm is proposed to estimate the global motions from the mixed motions based on the intrinsic property of camera adjustments. A two-stream 3D CNN framework is utilized for group activity recognition over the separated global and local motion patterns.
  arXiv  Detail & Related papers  (2020-07-13T10:15:44Z)
• Event detection in coarsely annotated sports videos via parallel multi receptive field 1D convolutions [14.30009544149561]
  In problems such as sports video analytics, it is difficult to obtain accurate frame level annotations and exact event duration. We propose the task of event detection in coarsely annotated videos. We introduce a multi-tower temporal convolutional network architecture for the proposed task.
  arXiv  Detail & Related papers  (2020-04-13T19:51:25Z)
• Unsupervised Temporal Feature Aggregation for Event Detection in Unstructured Sports Videos [10.230408415438966]
  We study the case of event detection in sports videos for unstructured environments with arbitrary camera angles. We identify and solve two major problems: unsupervised identification of players in an unstructured setting and generalization of the trained models to pose variations due to arbitrary shooting angles.
  arXiv  Detail & Related papers  (2020-02-19T10:24:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.