SportsHHI: A Dataset for Human-Human Interaction Detection in Sports Videos

• URL: http://arxiv.org/abs/2404.04565v1
• Date: Sat, 6 Apr 2024 09:13:03 GMT
• Title: SportsHHI: A Dataset for Human-Human Interaction Detection in Sports Videos
• Authors: Tao Wu, Runyu He, Gangshan Wu, Limin Wang,
• Abstract summary: We propose a new video visual relation detection task: video human-human interaction detection. SportsHHI contains 34 high-level interaction classes from basketball and volleyball sports. We conduct extensive experiments to reveal the key factors for a successful human-human interaction detector.
• Score: 43.536874272236986
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Video-based visual relation detection tasks, such as video scene graph generation, play important roles in fine-grained video understanding. However, current video visual relation detection datasets have two main limitations that hinder the progress of research in this area. First, they do not explore complex human-human interactions in multi-person scenarios. Second, the relation types of existing datasets have relatively low-level semantics and can be often recognized by appearance or simple prior information, without the need for detailed spatio-temporal context reasoning. Nevertheless, comprehending high-level interactions between humans is crucial for understanding complex multi-person videos, such as sports and surveillance videos. To address this issue, we propose a new video visual relation detection task: video human-human interaction detection, and build a dataset named SportsHHI for it. SportsHHI contains 34 high-level interaction classes from basketball and volleyball sports. 118,075 human bounding boxes and 50,649 interaction instances are annotated on 11,398 keyframes. To benchmark this, we propose a two-stage baseline method and conduct extensive experiments to reveal the key factors for a successful human-human interaction detector. We hope that SportsHHI can stimulate research on human interaction understanding in videos and promote the development of spatio-temporal context modeling techniques in video visual relation detection.

Related papers

• Human-Object Interaction Prediction in Videos through Gaze Following [9.61701724661823]
  We design a framework to detect current HOIs and anticipate future HOIs in videos. We propose to leverage human information since people often fixate on an object before interacting with it. Our model is trained and validated on the VidHOI dataset, which contains videos capturing daily life.
  arXiv  Detail & Related papers  (2023-06-06T11:36:14Z)
• How Would The Viewer Feel? Estimating Wellbeing From Video Scenarios [73.24092762346095]
  We introduce two large-scale datasets with over 60,000 videos annotated for emotional response and subjective wellbeing. The Video Cognitive Empathy dataset contains annotations for distributions of fine-grained emotional responses, allowing models to gain a detailed understanding of affective states. The Video to Valence dataset contains annotations of relative pleasantness between videos, which enables predicting a continuous spectrum of wellbeing.
  arXiv  Detail & Related papers  (2022-10-18T17:58:25Z)
• Weakly Supervised Human-Object Interaction Detection in Video via Contrastive Spatiotemporal Regions [81.88294320397826]
  A system does not know what human-object interactions are present in a video as or the actual location of the human and object. We introduce a dataset comprising over 6.5k videos with human-object interaction that have been curated from sentence captions. We demonstrate improved performance over weakly supervised baselines adapted to our annotations on our video dataset.
  arXiv  Detail & Related papers  (2021-10-07T15:30:18Z)
• Spatio-Temporal Interaction Graph Parsing Networks for Human-Object Interaction Recognition [55.7731053128204]
  In given video-based Human-Object Interaction scene, modeling thetemporal relationship between humans and objects are the important cue to understand the contextual information presented in the video. With the effective-temporal relationship modeling, it is possible not only to uncover contextual information in each frame but also directly capture inter-time dependencies. The full use of appearance features, spatial location and the semantic information are also the key to improve the video-based Human-Object Interaction recognition performance.
  arXiv  Detail & Related papers  (2021-08-19T11:57:27Z)
• LIGHTEN: Learning Interactions with Graph and Hierarchical TEmporal Networks for HOI in videos [13.25502885135043]
  Analyzing the interactions between humans and objects from a video includes identification of relationships between humans and the objects present in the video. We present a hierarchical approach, LIGHTEN, to learn visual features to effectively capture truth at multiple granularities in a video. We achieve state-of-the-art results in human-object interaction detection (88.9% and 92.6%) and anticipation tasks of CAD-120 and competitive results on image based HOI detection in V-COCO.
  arXiv  Detail & Related papers  (2020-12-17T05:44:07Z)
• Long Short-Term Relation Networks for Video Action Detection [155.13392337831166]
  Long Short-Term Relation Networks (LSTR) are presented in this paper. LSTR aggregates and propagates relation to augment features for video action detection. Extensive experiments are conducted on four benchmark datasets.
  arXiv  Detail & Related papers  (2020-03-31T10:02:51Z)
• Learning Human-Object Interaction Detection using Interaction Points [140.0200950601552]
  We propose a novel fully-convolutional approach that directly detects the interactions between human-object pairs. Our network predicts interaction points, which directly localize and classify the inter-action. Experiments are performed on two popular benchmarks: V-COCO and HICO-DET.
  arXiv  Detail & Related papers  (2020-03-31T08:42:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.