Skeleton-based Relational Reasoning for Group Activity Analysis

• URL: http://arxiv.org/abs/2011.05653v3
• Date: Fri, 8 Oct 2021 02:05:35 GMT
• Title: Skeleton-based Relational Reasoning for Group Activity Analysis
• Authors: Mauricio Perez, Jun Liu and Alex C. Kot
• Abstract summary: We leverage the skeleton information to learn the interactions between the individuals straight from it. Our experiments demonstrate the potential of skeleton-based approaches for modeling multi-person interactions.
• Score: 40.49389173100578
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Research on group activity recognition mostly leans on the standard two-stream approach (RGB and Optical Flow) as their input features. Few have explored explicit pose information, with none using it directly to reason about the persons interactions. In this paper, we leverage the skeleton information to learn the interactions between the individuals straight from it. With our proposed method GIRN, multiple relationship types are inferred from independent modules, that describe the relations between the body joints pair-by-pair. Additionally to the joints relations, we also experiment with the previously unexplored relationship between individuals and relevant objects (e.g. volleyball). The individuals distinct relations are then merged through an attention mechanism, that gives more importance to those individuals more relevant for distinguishing the group activity. We evaluate our method in the Volleyball dataset, obtaining competitive results to the state-of-the-art. Our experiments demonstrate the potential of skeleton-based approaches for modeling multi-person interactions.

Related papers

• Relation Learning and Aggregate-attention for Multi-person Motion Prediction [13.052342503276936]
  Multi-person motion prediction considers not just the skeleton structures or human trajectories but also the interactions between others. Previous methods often overlook that the joints relations within an individual (intra-relation) and interactions among groups (inter-relation) are distinct types of representations. We introduce a new collaborative framework for multi-person motion prediction that explicitly modeling these relations.
  arXiv  Detail & Related papers  (2024-11-06T07:48:30Z)
• Visual-Geometric Collaborative Guidance for Affordance Learning [63.038406948791454]
  We propose a visual-geometric collaborative guided affordance learning network that incorporates visual and geometric cues. Our method outperforms the representative models regarding objective metrics and visual quality.
  arXiv  Detail & Related papers  (2024-10-15T07:35:51Z)
• Exploring Self- and Cross-Triplet Correlations for Human-Object Interaction Detection [38.86053346974547]
  We propose to explore Self- and Cross-Triplet Correlations for HOI detection. Specifically, we regard each triplet proposal as a graph where Human, Object represent nodes and Action indicates edge. Also, we try to explore cross-triplet dependencies by jointly considering instance-level, semantic-level, and layout-level relations.
  arXiv  Detail & Related papers  (2024-01-11T05:38:24Z)
• Group Activity Recognition via Dynamic Composition and Interaction [8.83578086094184]
  We propose DynamicFormer with Dynamic composition Module (DcM) and Dynamic interaction Module (DiM) to model relations and locations of persons. Our findings on group composition and human-object interaction inspire our core idea. We conduct extensive experiments on two public datasets and show that our method achieves state-of-the-art.
  arXiv  Detail & Related papers  (2023-05-09T16:18:18Z)
• Rethinking Trajectory Prediction via "Team Game" [118.59480535826094]
  We present a novel formulation for multi-agent trajectory prediction, which explicitly introduces the concept of interactive group consensus. On two multi-agent settings, i.e. team sports and pedestrians, the proposed framework consistently achieves superior performance compared to existing methods.
  arXiv  Detail & Related papers  (2022-10-17T07:16:44Z)
• Multi-Agent Imitation Learning with Copulas [102.27052968901894]
  Multi-agent imitation learning aims to train multiple agents to perform tasks from demonstrations by learning a mapping between observations and actions. In this paper, we propose to use copula, a powerful statistical tool for capturing dependence among random variables, to explicitly model the correlation and coordination in multi-agent systems. Our proposed model is able to separately learn marginals that capture the local behavioral patterns of each individual agent, as well as a copula function that solely and fully captures the dependence structure among agents.
  arXiv  Detail & Related papers  (2021-07-10T03:49:41Z)
• Information Interaction Profile of Choice Adoption [2.9972063833424216]
  We introduce an efficient method to infer the entities interaction network and its evolution according to the temporal distance separating interacting entities. The interaction profile allows characterizing the mechanisms of the interaction processes. We show that the effect of a combination of exposures on a user is more than the sum of each exposure's independent effect--there is an interaction.
  arXiv  Detail & Related papers  (2021-04-28T10:42:25Z)
• Learning Relation Prototype from Unlabeled Texts for Long-tail Relation Extraction [84.64435075778988]
  We propose a general approach to learn relation prototypes from unlabeled texts. We learn relation prototypes as an implicit factor between entities. We conduct experiments on two publicly available datasets: New York Times and Google Distant Supervision.
  arXiv  Detail & Related papers  (2020-11-27T06:21:12Z)
• Learning to Decouple Relations: Few-Shot Relation Classification with Entity-Guided Attention and Confusion-Aware Training [49.9995628166064]
  We propose CTEG, a model equipped with two mechanisms to learn to decouple easily-confused relations. On the one hand, an EGA mechanism is introduced to guide the attention to filter out information causing confusion. On the other hand, a Confusion-Aware Training (CAT) method is proposed to explicitly learn to distinguish relations.
  arXiv  Detail & Related papers  (2020-10-21T11:07:53Z)
• Interactions in information spread: quantification and interpretation using stochastic block models [3.5450828190071655]
  In social networks, users' behavior results from the people they interact with, news in their feed, or trending topics. Here, we propose a new model, the Interactive Mixed Membership Block Model (IMMSBM), which investigates the role of interactions between entities. In inference tasks, taking them into account leads to average relative changes with respect to non-interactive models of up to 150% in the probability of an outcome.
  arXiv  Detail & Related papers  (2020-04-09T14:22:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.