An Open-source Benchmark of Deep Learning Models for Audio-visual Apparent and Self-reported Personality Recognition

• URL: http://arxiv.org/abs/2210.09138v2
• Date: Tue, 6 Feb 2024 04:58:07 GMT
• Title: An Open-source Benchmark of Deep Learning Models for Audio-visual Apparent and Self-reported Personality Recognition
• Authors: Rongfan Liao and Siyang Song and Hatice Gunes
• Abstract summary: Personality determines a wide variety of human daily and working behaviours, and is crucial for understanding human internal and external states. In recent years, a large number of automatic personality computing approaches have been developed to predict either the apparent personality or self-reported personality of the subject based on non-verbal audio-visual behaviours. In the absence of a standardized benchmark with consistent experimental settings, it is not only impossible to fairly compare the real performances of these personality computing models but also makes them difficult to be reproduced. We present the first reproducible audio-visual benchmarking framework to provide a fair and consistent evaluation of eight existing personality computing models and
• Score: 10.59440995582639
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Personality determines a wide variety of human daily and working behaviours, and is crucial for understanding human internal and external states. In recent years, a large number of automatic personality computing approaches have been developed to predict either the apparent personality or self-reported personality of the subject based on non-verbal audio-visual behaviours. However, the majority of them suffer from complex and dataset-specific pre-processing steps and model training tricks. In the absence of a standardized benchmark with consistent experimental settings, it is not only impossible to fairly compare the real performances of these personality computing models but also makes them difficult to be reproduced. In this paper, we present the first reproducible audio-visual benchmarking framework to provide a fair and consistent evaluation of eight existing personality computing models (e.g., audio, visual and audio-visual) and seven standard deep learning models on both self-reported and apparent personality recognition tasks. Building upon a set of benchmarked models, we also investigate the impact of two previously-used long-term modelling strategies for summarising short-term/frame-level predictions on personality computing results. The results conclude: (i) apparent personality traits, inferred from facial behaviours by most benchmarked deep learning models, show more reliability than self-reported ones; (ii) visual models frequently achieved superior performances than audio models on personality recognition; (iii) non-verbal behaviours contribute differently in predicting different personality traits; and (iv) our reproduced personality computing models generally achieved worse performances than their original reported results. Our benchmark is publicly available at \url{https://github.com/liaorongfan/DeepPersonality}.

Related papers

• LLMvsSmall Model? Large Language Model Based Text Augmentation Enhanced Personality Detection Model [58.887561071010985]
  Personality detection aims to detect one's personality traits underlying in social media posts. Most existing methods learn post features directly by fine-tuning the pre-trained language models. We propose a large language model (LLM) based text augmentation enhanced personality detection model.
  arXiv  Detail & Related papers  (2024-03-12T12:10:18Z)
• PsyCoT: Psychological Questionnaire as Powerful Chain-of-Thought for Personality Detection [50.66968526809069]
  We propose a novel personality detection method, called PsyCoT, which mimics the way individuals complete psychological questionnaires in a multi-turn dialogue manner. Our experiments demonstrate that PsyCoT significantly improves the performance and robustness of GPT-3.5 in personality detection.
  arXiv  Detail & Related papers  (2023-10-31T08:23:33Z)
• Preserving Knowledge Invariance: Rethinking Robustness Evaluation of Open Information Extraction [50.62245481416744]
  We present the first benchmark that simulates the evaluation of open information extraction models in the real world. We design and annotate a large-scale testbed in which each example is a knowledge-invariant clique. By further elaborating the robustness metric, a model is judged to be robust if its performance is consistently accurate on the overall cliques.
  arXiv  Detail & Related papers  (2023-05-23T12:05:09Z)
• Analysing the Impact of Audio Quality on the Use of Naturalistic Long-Form Recordings for Infant-Directed Speech Research [62.997667081978825]
  Modelling of early language acquisition aims to understand how infants bootstrap their language skills. Recent developments have enabled the use of more naturalistic training data for computational models. It is currently unclear how the sound quality could affect analyses and modelling experiments conducted on such data.
  arXiv  Detail & Related papers  (2023-05-03T08:25:37Z)
• Personality-aware Human-centric Multimodal Reasoning: A New Task, Dataset and Baselines [32.82738983843281]
  We introduce a new task called Personality-aware Human-centric Multimodal Reasoning (PHMR) (T1) The goal of the task is to forecast the future behavior of a particular individual using multimodal information from past instances, while integrating personality factors. The experimental results demonstrate that incorporating personality traits enhances human-centric multimodal reasoning performance.
  arXiv  Detail & Related papers  (2023-04-05T09:09:10Z)
• Domain-specific Learning of Multi-scale Facial Dynamics for Apparent Personality Traits Prediction [3.19935268158731]
  We propose a novel video-based automatic personality traits recognition approach. It consists of: (1) a textbfdomain-specific facial behavior modelling module that extracts personality-related multi-scale short-term human facial behavior features; (2) a textbflong-term behavior modelling module that summarizes all short-term features of a video as a long-term/video-level personality representation; and (3) a textbfmulti-task personality traits prediction module that models underlying relationship among all traits and jointly predict them based on the video-level personality representation.
  arXiv  Detail & Related papers  (2022-09-09T07:08:55Z)
• CIAO! A Contrastive Adaptation Mechanism for Non-Universal Facial Expression Recognition [80.07590100872548]
  We propose Contrastive Inhibitory Adaptati On (CIAO), a mechanism that adapts the last layer of facial encoders to depict specific affective characteristics on different datasets. CIAO presents an improvement in facial expression recognition performance over six different datasets with very unique affective representations.
  arXiv  Detail & Related papers  (2022-08-10T15:46:05Z)
• Fact sheet: Automatic Self-Reported Personality Recognition Track [0.6091702876917281]
  We analysed the correlation between the given metadata and the self-assigned personality trait scores. We compared the performance of this informed baseline with models based on state-of-the-art visual, linguistic and audio features.
  arXiv  Detail & Related papers  (2022-07-22T11:30:11Z)
• Self-supervised models of audio effectively explain human cortical responses to speech [71.57870452667369]
  We capitalize on the progress of self-supervised speech representation learning to create new state-of-the-art models of the human auditory system. We show that these results show that self-supervised models effectively capture the hierarchy of information relevant to different stages of speech processing in human cortex.
  arXiv  Detail & Related papers  (2022-05-27T22:04:02Z)
• Multitask Learning for Emotion and Personality Detection [17.029426018676997]
  We build on the known correlation between personality traits and emotional behaviors, and propose a novel multitask learning framework, SoGMTL. Our more computationally efficient CNN-based multitask model achieves the state-of-the-art performance across multiple famous personality and emotion datasets.
  arXiv  Detail & Related papers  (2021-01-07T03:09:55Z)
• Personality Trait Detection Using Bagged SVM over BERT Word Embedding Ensembles [10.425280599592865]
  We present a novel deep learning-based approach for automated personality detection from text. We leverage state of the art advances in natural language understanding, namely the BERT language model to extract contextualized word embeddings. Our model outperforms the previous state of the art by 1.04% and, at the same time is significantly more computationally efficient to train.
  arXiv  Detail & Related papers  (2020-10-03T09:25:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.