REACT2023: the first Multi-modal Multiple Appropriate Facial Reaction Generation Challenge

• URL: http://arxiv.org/abs/2306.06583v1
• Date: Sun, 11 Jun 2023 04:15:56 GMT
• Title: REACT2023: the first Multi-modal Multiple Appropriate Facial Reaction Generation Challenge
• Authors: Siyang Song, Micol Spitale, Cheng Luo, German Barquero, Cristina Palmero, Sergio Escalera, Michel Valstar, Tobias Baur, Fabien Ringeval, Elisabeth Andre and Hatice Gunes
• Abstract summary: The Multi-modal Multiple Appropriate Facial Reaction Generation Challenge (REACT2023) is the first competition event focused on evaluating multimedia processing and machine learning techniques for generating human-appropriate facial reactions in various dyadic interaction scenarios. The goal of the challenge is to provide the first benchmark test set for multi-modal information processing and to foster collaboration among the audio, visual, and audio-visual affective computing communities.
• Score: 28.777465429875303
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: The Multi-modal Multiple Appropriate Facial Reaction Generation Challenge (REACT2023) is the first competition event focused on evaluating multimedia processing and machine learning techniques for generating human-appropriate facial reactions in various dyadic interaction scenarios, with all participants competing strictly under the same conditions. The goal of the challenge is to provide the first benchmark test set for multi-modal information processing and to foster collaboration among the audio, visual, and audio-visual affective computing communities, to compare the relative merits of the approaches to automatic appropriate facial reaction generation under different spontaneous dyadic interaction conditions. This paper presents: (i) novelties, contributions and guidelines of the REACT2023 challenge; (ii) the dataset utilized in the challenge; and (iii) the performance of baseline systems on the two proposed sub-challenges: Offline Multiple Appropriate Facial Reaction Generation and Online Multiple Appropriate Facial Reaction Generation, respectively. The challenge baseline code is publicly available at \url{https://github.com/reactmultimodalchallenge/baseline_react2023}.

Related papers

• Second Edition FRCSyn Challenge at CVPR 2024: Face Recognition Challenge in the Era of Synthetic Data [104.45155847778584]
  This paper presents an overview of the 2nd edition of the Face Recognition Challenge in the Era of Synthetic Data (FRCSyn) FRCSyn aims to investigate the use of synthetic data in face recognition to address current technological limitations.
  arXiv  Detail & Related papers  (2024-04-16T08:15:10Z)
• REACT 2024: the Second Multiple Appropriate Facial Reaction Generation Challenge [36.84914349494818]
  In dyadic interactions, humans communicate their intentions and state of mind using verbal and non-verbal cues. How to develop a machine learning (ML) model that can automatically generate multiple appropriate, diverse, realistic and synchronised human facial reactions is a challenging task. This paper presents the guidelines of the REACT 2024 challenge and the dataset utilized in the challenge.
  arXiv  Detail & Related papers  (2024-01-10T14:01:51Z)
• ReactFace: Online Multiple Appropriate Facial Reaction Generation in Dyadic Interactions [46.66378299720377]
  In dyadic interaction, predicting the listener's facial reactions is challenging as different reactions could be appropriate in response to the same speaker's behaviour. This paper reformulates the task as an extrapolation or prediction problem, and proposes a novel framework (called ReactFace) to generate multiple different but appropriate facial reactions.
  arXiv  Detail & Related papers  (2023-05-25T05:55:53Z)
• Reversible Graph Neural Network-based Reaction Distribution Learning for Multiple Appropriate Facial Reactions Generation [22.579200870471475]
  This paper proposes the first multiple appropriate facial reaction generation framework. It re-formulates the one-to-many mapping facial reaction generation problem as a one-to-one mapping problem. Experimental results demonstrate that our approach outperforms existing models in generating more appropriate, realistic, and synchronized facial reactions.
  arXiv  Detail & Related papers  (2023-05-24T15:56:26Z)
• Modeling Complex Event Scenarios via Simple Entity-focused Questions [58.16787028844743]
  We propose a question-guided generation framework that models events in complex scenarios as answers to questions about participants. At any step in the generation process, the framework uses the previously generated events as context, but generates the next event as an answer to one of three questions. Our empirical evaluation shows that this question-guided generation provides better coverage of participants, diverse events within a domain, and comparable perplexities for modeling event sequences.
  arXiv  Detail & Related papers  (2023-02-14T15:48:56Z)
• Face-to-Face Contrastive Learning for Social Intelligence Question-Answering [55.90243361923828]
  multimodal methods have set the state of the art on many tasks, but have difficulty modeling the complex face-to-face conversational dynamics. We propose Face-to-Face Contrastive Learning (F2F-CL), a graph neural network designed to model social interactions. We experimentally evaluated the challenging Social-IQ dataset and show state-of-the-art results.
  arXiv  Detail & Related papers  (2022-07-29T20:39:44Z)
• HeterMPC: A Heterogeneous Graph Neural Network for Response Generation in Multi-Party Conversations [76.64792382097724]
  We present HeterMPC, a graph-based neural network for response generation in multi-party conversations (MPCs) HeterMPC models the semantics of utterances and interlocutors simultaneously with two types of nodes in a graph. Through multi-hop updating, HeterMPC can adequately utilize the structural knowledge of conversations for response generation.
  arXiv  Detail & Related papers  (2022-03-16T09:50:32Z)
• HiFaceGAN: Face Renovation via Collaborative Suppression and Replenishment [63.333407973913374]
  "Face Renovation"(FR) is a semantic-guided generation problem. "HiFaceGAN" is a multi-stage framework containing several nested CSR units. experiments on both synthetic and real face images have verified the superior performance of HiFaceGAN.
  arXiv  Detail & Related papers  (2020-05-11T11:33:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.