Hunting Group Clues with Transformers for Social Group Activity Recognition

• URL: http://arxiv.org/abs/2207.05254v1
• Date: Tue, 12 Jul 2022 01:46:46 GMT
• Title: Hunting Group Clues with Transformers for Social Group Activity Recognition
• Authors: Masato Tamura, Rahul Vishwakarma, Ravigopal Vennelakanti
• Abstract summary: Social group activity recognition requires recognizing multiple sub-group activities and identifying group members. Most existing methods tackle both tasks by refining region features and then summarizing them into activity features. We propose to leverage attention modules in transformers to generate effective social group features. Our method is designed in such a way that the attention modules identify and then aggregate features relevant to social group activities.
• Score: 3.1061678033205635
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper presents a novel framework for social group activity recognition. As an expanded task of group activity recognition, social group activity recognition requires recognizing multiple sub-group activities and identifying group members. Most existing methods tackle both tasks by refining region features and then summarizing them into activity features. Such heuristic feature design renders the effectiveness of features susceptible to incomplete person localization and disregards the importance of scene contexts. Furthermore, region features are sub-optimal to identify group members because the features may be dominated by those of people in the regions and have different semantics. To overcome these drawbacks, we propose to leverage attention modules in transformers to generate effective social group features. Our method is designed in such a way that the attention modules identify and then aggregate features relevant to social group activities, generating an effective feature for each social group. Group member information is embedded into the features and thus accessed by feed-forward networks. The outputs of feed-forward networks represent groups so concisely that group members can be identified with simple Hungarian matching between groups and individuals. Experimental results show that our method outperforms state-of-the-art methods on the Volleyball and Collective Activity datasets.

Related papers

• Design and Analysis of Efficient Attention in Transformers for Social Group Activity Recognition [3.75292409381511]
  We propose leveraging attention modules in transformers to generate social group features. Multiple embeddings are used to aggregate features for a social group, each of which is assigned to a group member without duplication. The proposed method achieves state-of-the-art performance and verify that the proposed attention designs are highly effective on social group activity recognition.
  arXiv  Detail & Related papers  (2024-04-15T17:40:23Z)
• Towards More Practical Group Activity Detection: A New Benchmark and Model [61.39427407758131]
  Group activity detection (GAD) is the task of identifying members of each group and classifying the activity of the group at the same time in a video. We present a new dataset, dubbed Caf'e, which presents more practical scenarios and metrics. We also propose a new GAD model that deals with an unknown number of groups and latent group members efficiently and effectively.
  arXiv  Detail & Related papers  (2023-12-05T16:48:17Z)
• Group Activity Recognition in Computer Vision: A Comprehensive Review, Challenges, and Future Perspectives [0.0]
  Group activity recognition is a hot topic in computer vision. Recognizing activities through group relationships plays a vital role in group activity recognition. This work examines the progress in technology for recognizing group activities.
  arXiv  Detail & Related papers  (2023-07-25T14:44:41Z)
• Ranking-based Group Identification via Factorized Attention on Social Tripartite Graph [68.08590487960475]
  We propose a novel GNN-based framework named Contextualized Factorized Attention for Group identification (CFAG) We devise tripartite graph convolution layers to aggregate information from different types of neighborhoods among users, groups, and items. To cope with the data sparsity issue, we devise a novel propagation augmentation layer, which is based on our proposed factorized attention mechanism.
  arXiv  Detail & Related papers  (2022-11-02T01:42:20Z)
• Attentive pooling for Group Activity Recognition [23.241686027269928]
  In group activity recognition, hierarchical framework is widely adopted to represent the relationships between individuals and their corresponding group. We propose a new contextual pooling scheme, named attentive pooling, which enables the weighted information transition from individual actions to group activity.
  arXiv  Detail & Related papers  (2022-08-31T13:26:39Z)
• Towards Group Robustness in the presence of Partial Group Labels [61.33713547766866]
  spurious correlations between input samples and the target labels wrongly direct the neural network predictions. We propose an algorithm that optimize for the worst-off group assignments from a constraint set. We show improvements in the minority group's performance while preserving overall aggregate accuracy across groups.
  arXiv  Detail & Related papers  (2022-01-10T22:04:48Z)
• Learning Multi-Attention Context Graph for Group-Based Re-Identification [214.84551361855443]
  Learning to re-identify or retrieve a group of people across non-overlapped camera systems has important applications in video surveillance. In this work, we consider employing context information for identifying groups of people, i.e., group re-id. We propose a novel unified framework based on graph neural networks to simultaneously address the group-based re-id tasks.
  arXiv  Detail & Related papers  (2021-04-29T09:57:47Z)
• Overcoming Data Sparsity in Group Recommendation [52.00998276970403]
  Group recommender systems should be able to accurately learn not only users' personal preferences but also preference aggregation strategy. In this paper, we take Bipartite Graphding Model (BGEM), the self-attention mechanism and Graph Convolutional Networks (GCNs) as basic building blocks to learn group and user representations in a unified way.
  arXiv  Detail & Related papers  (2020-10-02T07:11:19Z)
• Social Adaptive Module for Weakly-supervised Group Activity Recognition [143.68241396839062]
  This paper presents a new task named weakly-supervised group activity recognition (GAR) It differs from conventional GAR tasks in that only video-level labels are available, yet the important persons within each frame are not provided even in the training data. This eases us to collect and annotate a large-scale NBA dataset and thus raise new challenges to GAR.
  arXiv  Detail & Related papers  (2020-07-18T16:40:55Z)
• Joint Learning of Social Groups, Individuals Action and Sub-group Activities in Videos [23.15064911470468]
  We propose an end-to-end trainable framework for the social task. Our main contributions are: i) we propose an end-to-end trainable framework for the social task; ii) our proposed method sets the state-of-the-art results on two widely adopted benchmarks for the traditional group recognition activity task.
  arXiv  Detail & Related papers  (2020-07-06T10:42:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.