MultiMediate'23: Engagement Estimation and Bodily Behaviour Recognition in Social Interactions

• URL: http://arxiv.org/abs/2308.08256v1
• Date: Wed, 16 Aug 2023 09:47:52 GMT
• Title: MultiMediate'23: Engagement Estimation and Bodily Behaviour Recognition in Social Interactions
• Abstract summary: We address two key human social behaviour analysis tasks for the first time in a controlled challenge: engagement estimation and bodily behaviour recognition in social interactions. This paper describes the MultiMediate'23 challenge and presents novel sets of annotations for both tasks.
• Score: 42.94144353625103
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Automatic analysis of human behaviour is a fundamental prerequisite for the creation of machines that can effectively interact with- and support humans in social interactions. In MultiMediate'23, we address two key human social behaviour analysis tasks for the first time in a controlled challenge: engagement estimation and bodily behaviour recognition in social interactions. This paper describes the MultiMediate'23 challenge and presents novel sets of annotations for both tasks. For engagement estimation we collected novel annotations on the NOvice eXpert Interaction (NOXI) database. For bodily behaviour recognition, we annotated test recordings of the MPIIGroupInteraction corpus with the BBSI annotation scheme. In addition, we present baseline results for both challenge tasks.

Related papers

• Does Theory of Mind Improvement Really Benefit Human-AI Interactions? Empirical Findings from Interactive Evaluations [80.39504840645687]
  Existing benchmarks measure ToM capability improvement through story-reading, multiple-choice questions from a third-person perspective.<n>This paper proposes a new paradigm of interactive ToM evaluation with both perspective and metric shifts.<n>Our findings reveal that improvements on static benchmarks do not always translate to better performance in dynamic HAI interactions.
  arXiv  Detail & Related papers  (2026-04-28T15:38:31Z)
• SocialOmni: Benchmarking Audio-Visual Social Interactivity in Omni Models [86.19617358080016]
  Social Omni is a benchmark that operationalizes the evaluation of conversational interactivity across three core dimensions.<n>Social Omni features 2,000 perception samples and a quality-controlled diagnostic set of 209 interaction-generation instances.<n>Our analysis reveals a pronounced decoupling between a model's perceptual accuracy and its ability to generate contextually appropriate interruptions.
  arXiv  Detail & Related papers  (2026-03-17T17:58:44Z)
• AntEval: Evaluation of Social Interaction Competencies in LLM-Driven Agents [65.16893197330589]
  Large Language Models (LLMs) have demonstrated their ability to replicate human behaviors across a wide range of scenarios. However, their capability in handling complex, multi-character social interactions has yet to be fully explored. We introduce the Multi-Agent Interaction Evaluation Framework (AntEval), encompassing a novel interaction framework and evaluation methods.
  arXiv  Detail & Related papers  (2024-01-12T11:18:00Z)
• Disentangled Interaction Representation for One-Stage Human-Object Interaction Detection [70.96299509159981]
  Human-Object Interaction (HOI) detection is a core task for human-centric image understanding. Recent one-stage methods adopt a transformer decoder to collect image-wide cues that are useful for interaction prediction. Traditional two-stage methods benefit significantly from their ability to compose interaction features in a disentangled and explainable manner.
  arXiv  Detail & Related papers  (2023-12-04T08:02:59Z)
• Automatic Context-Driven Inference of Engagement in HMI: A Survey [6.479224589451863]
  This paper presents a survey on engagement inference for human-machine interaction. It entails interdisciplinary definition, engagement components and factors, publicly available datasets, ground truth assessment, and most commonly used features and methods. It serves as a guide for the development of future human-machine interaction interfaces with reliable context-aware engagement inference capability.
  arXiv  Detail & Related papers  (2022-09-30T10:46:13Z)
• Bodily Behaviors in Social Interaction: Novel Annotations and State-of-the-Art Evaluation [0.0]
  We present BBSI, the first set of annotations of complex Bodily Behaviors embedded in continuous Social Interactions. Based on previous work in psychology, we manually annotated 26 hours of spontaneous human behavior. We adapt the Pyramid Dilated Attention Network (PDAN), a state-of-the-art approach for human action detection.
  arXiv  Detail & Related papers  (2022-07-26T11:24:00Z)
• Co-Located Human-Human Interaction Analysis using Nonverbal Cues: A Survey [71.43956423427397]
  We aim to identify the nonverbal cues and computational methodologies resulting in effective performance. This survey differs from its counterparts by involving the widest spectrum of social phenomena and interaction settings. Some major observations are: the most often used nonverbal cue, computational method, interaction environment, and sensing approach are speaking activity, support vector machines, and meetings composed of 3-4 persons equipped with microphones and cameras, respectively.
  arXiv  Detail & Related papers  (2022-07-20T13:37:57Z)
• Self-supervised Social Relation Representation for Human Group Detection [18.38523753680367]
  We propose a new two-stage multi-head framework for human group detection. In the first stage, we propose a human behavior simulator head to learn the social relation feature embedding. In the second stage, based on the social relation embedding, we develop a self-attention inspired network for human group detection.
  arXiv  Detail & Related papers  (2022-03-08T04:26:07Z)
• Didn't see that coming: a survey on non-verbal social human behavior forecasting [47.99589136455976]
  Non-verbal social human behavior forecasting has increasingly attracted the interest of the research community in recent years. Its direct applications to human-robot interaction and socially-aware human motion generation make it a very attractive field. We define the behavior forecasting problem for multiple interactive agents in a generic way that aims at unifying the fields of social signals prediction and human motion forecasting.
  arXiv  Detail & Related papers  (2022-03-04T18:25:30Z)
• Cascaded Human-Object Interaction Recognition [175.60439054047043]
  We introduce a cascade architecture for a multi-stage, coarse-to-fine HOI understanding. At each stage, an instance localization network progressively refines HOI proposals and feeds them into an interaction recognition network. With our carefully-designed human-centric relation features, these two modules work collaboratively towards effective interaction understanding.
  arXiv  Detail & Related papers  (2020-03-09T17:05:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.