Conversation Group Detection With Spatio-Temporal Context

• URL: http://arxiv.org/abs/2206.02559v1
• Date: Thu, 2 Jun 2022 08:05:02 GMT
• Title: Conversation Group Detection With Spatio-Temporal Context
• Authors: Stephanie Tan, David M.J. Tax, Hayley Hung
• Abstract summary: We propose an approach for detecting conversation groups in social scenarios like cocktail parties and networking events. We posit the detection of conversation groups as a learning problem that could benefit from leveraging the spatial context of the surroundings. This motivates our approach which consists of a dynamic LSTM-based deep learning model that predicts continuous pairwise affinity values.
• Score: 11.288403109735544
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this work, we propose an approach for detecting conversation groups in social scenarios like cocktail parties and networking events, from overhead camera recordings. We posit the detection of conversation groups as a learning problem that could benefit from leveraging the spatial context of the surroundings, and the inherent temporal context in interpersonal dynamics which is reflected in the temporal dynamics in human behavior signals, an aspect that has not been addressed in recent prior works. This motivates our approach which consists of a dynamic LSTM-based deep learning model that predicts continuous pairwise affinity values indicating how likely two people are in the same conversation group. These affinity values are also continuous in time, since relationships and group membership do not occur instantaneously, even though the ground truths of group membership are binary. Using the predicted affinity values, we apply a graph clustering method based on Dominant Set extraction to identify the conversation groups. We benchmark the proposed method against established methods on multiple social interaction datasets. Our results showed that the proposed method improves group detection performance in data that has more temporal granularity in conversation group labels. Additionally, we provide an analysis in the predicted affinity values in relation to the conversation group detection. Finally, we demonstrate the usability of the predicted affinity values in a forecasting framework to predict group membership for a given forecast horizon.

Related papers

• Exploiting temporal information to detect conversational groups in videos and predict the next speaker [2.7981106665946944]
  This paper aims at detecting F formations in video sequences and predicting the next speaker in a group conversation. We rely on measuring the engagement level of people as a feature of group belonging. Experiments on the MatchNMingle dataset led to 85% true positives in group detection and 98% accuracy in predicting the next speaker.
  arXiv  Detail & Related papers  (2024-08-29T09:41:36Z)
• Redefining Event Types and Group Evolution in Temporal Data [0.16385815610837165]
  In temporal data, the predominant approach for characterizing group evolution has been through the identification of events" We think of events as archetypes" characterized by a unique combination of quantitative dimensions that we call facet extremities" We apply our framework to evolving groups from several face-to-face interaction datasets, showing it enables richer, more reliable characterization of group dynamics.
  arXiv  Detail & Related papers  (2024-03-11T14:39:24Z)
• Towards Efficient and Effective Deep Clustering with Dynamic Grouping and Prototype Aggregation [4.550555443103878]
  We present a novel end-to-end deep clustering framework with dynamic grouping and prototype aggregation, termed as DigPro. Specifically, the proposed dynamic grouping extends contrastive learning from instance-level to group-level, which is effective and efficient for timely updating groups. With an expectation-maximization framework, DigPro simultaneously takes advantage of compact intra-cluster connections, well-separated clusters, and efficient group updating during the self-supervised training.
  arXiv  Detail & Related papers  (2024-01-24T16:45:42Z)
• Parallel Reasoning Network for Human-Object Interaction Detection [53.422076419484945]
  We propose a new transformer-based method named Parallel Reasoning Network(PR-Net) PR-Net constructs two independent predictors for instance-level localization and relation-level understanding. Our PR-Net has achieved competitive results on HICO-DET and V-COCO benchmarks.
  arXiv  Detail & Related papers  (2023-01-09T17:00:34Z)
• Rethinking Trajectory Prediction via "Team Game" [118.59480535826094]
  We present a novel formulation for multi-agent trajectory prediction, which explicitly introduces the concept of interactive group consensus. On two multi-agent settings, i.e. team sports and pedestrians, the proposed framework consistently achieves superior performance compared to existing methods.
  arXiv  Detail & Related papers  (2022-10-17T07:16:44Z)
• Co-Located Human-Human Interaction Analysis using Nonverbal Cues: A Survey [71.43956423427397]
  We aim to identify the nonverbal cues and computational methodologies resulting in effective performance. This survey differs from its counterparts by involving the widest spectrum of social phenomena and interaction settings. Some major observations are: the most often used nonverbal cue, computational method, interaction environment, and sensing approach are speaking activity, support vector machines, and meetings composed of 3-4 persons equipped with microphones and cameras, respectively.
  arXiv  Detail & Related papers  (2022-07-20T13:37:57Z)
• Grouptron: Dynamic Multi-Scale Graph Convolutional Networks for Group-Aware Dense Crowd Trajectory Forecasting [30.885838880247263]
  Grouptron is a multi-scale dynamic forecasting framework that leverages pedestrian group detection and representation. Method achieves 9.3% decrease in final displacement error (FDE) compared with state-of-the-art methods on ETH/UCY benchmark datasets.
  arXiv  Detail & Related papers  (2021-09-29T01:22:25Z)
• Modelling Neighbor Relation in Joint Space-Time Graph for Video Correspondence Learning [53.74240452117145]
  This paper presents a self-supervised method for learning reliable visual correspondence from unlabeled videos. We formulate the correspondence as finding paths in a joint space-time graph, where nodes are grid patches sampled from frames, and are linked by two types of edges. Our learned representation outperforms the state-of-the-art self-supervised methods on a variety of visual tasks.
  arXiv  Detail & Related papers  (2021-09-28T05:40:01Z)
• CoADNet: Collaborative Aggregation-and-Distribution Networks for Co-Salient Object Detection [91.91911418421086]
  Co-Salient Object Detection (CoSOD) aims at discovering salient objects that repeatedly appear in a given query group containing two or more relevant images. One challenging issue is how to effectively capture co-saliency cues by modeling and exploiting inter-image relationships. We present an end-to-end collaborative aggregation-and-distribution network (CoADNet) to capture both salient and repetitive visual patterns from multiple images.
  arXiv  Detail & Related papers  (2020-11-10T04:28:11Z)
• Group Activity Prediction with Sequential Relational Anticipation Model [28.225918711577314]
  We propose a novel approach to predict group activities given the beginning frames with incomplete activity executions. For group activity prediction, the relation evolution of people's activity and their positions over time is an important cue for predicting group activity. Our model explicitly anticipates both activity features and positions by two graph auto-encoders, aiming to learn a discriminative group representation for group activity prediction.
  arXiv  Detail & Related papers  (2020-08-06T03:17:14Z)
• Predicting Temporal Sets with Deep Neural Networks [50.53727580527024]
  We propose an integrated solution based on the deep neural networks for temporal sets prediction. A unique perspective is to learn element relationship by constructing set-level co-occurrence graph. We design an attention-based module to adaptively learn the temporal dependency of elements and sets.
  arXiv  Detail & Related papers  (2020-06-20T03:29:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.