An Open-source Benchmark of Deep Learning Models for Audio-visual
Apparent and Self-reported Personality Recognition
- URL: http://arxiv.org/abs/2210.09138v2
- Date: Tue, 6 Feb 2024 04:58:07 GMT
- Title: An Open-source Benchmark of Deep Learning Models for Audio-visual
Apparent and Self-reported Personality Recognition
- Authors: Rongfan Liao and Siyang Song and Hatice Gunes
- Abstract summary: Personality determines a wide variety of human daily and working behaviours, and is crucial for understanding human internal and external states.
In recent years, a large number of automatic personality computing approaches have been developed to predict either the apparent personality or self-reported personality of the subject based on non-verbal audio-visual behaviours.
In the absence of a standardized benchmark with consistent experimental settings, it is not only impossible to fairly compare the real performances of these personality computing models but also makes them difficult to be reproduced.
We present the first reproducible audio-visual benchmarking framework to provide a fair and consistent evaluation of eight existing personality computing models and
- Score: 10.59440995582639
- License: http://creativecommons.org/publicdomain/zero/1.0/
- Abstract: Personality determines a wide variety of human daily and working behaviours,
and is crucial for understanding human internal and external states. In recent
years, a large number of automatic personality computing approaches have been
developed to predict either the apparent personality or self-reported
personality of the subject based on non-verbal audio-visual behaviours.
However, the majority of them suffer from complex and dataset-specific
pre-processing steps and model training tricks. In the absence of a
standardized benchmark with consistent experimental settings, it is not only
impossible to fairly compare the real performances of these personality
computing models but also makes them difficult to be reproduced. In this paper,
we present the first reproducible audio-visual benchmarking framework to
provide a fair and consistent evaluation of eight existing personality
computing models (e.g., audio, visual and audio-visual) and seven standard deep
learning models on both self-reported and apparent personality recognition
tasks. Building upon a set of benchmarked models, we also investigate the
impact of two previously-used long-term modelling strategies for summarising
short-term/frame-level predictions on personality computing results. The
results conclude: (i) apparent personality traits, inferred from facial
behaviours by most benchmarked deep learning models, show more reliability than
self-reported ones; (ii) visual models frequently achieved superior
performances than audio models on personality recognition; (iii) non-verbal
behaviours contribute differently in predicting different personality traits;
and (iv) our reproduced personality computing models generally achieved worse
performances than their original reported results. Our benchmark is publicly
available at \url{https://github.com/liaorongfan/DeepPersonality}.
Related papers
- LLMvsSmall Model? Large Language Model Based Text Augmentation Enhanced
Personality Detection Model [58.887561071010985]
Personality detection aims to detect one's personality traits underlying in social media posts.
Most existing methods learn post features directly by fine-tuning the pre-trained language models.
We propose a large language model (LLM) based text augmentation enhanced personality detection model.
arXiv Detail & Related papers (2024-03-12T12:10:18Z) - PsyCoT: Psychological Questionnaire as Powerful Chain-of-Thought for
Personality Detection [50.66968526809069]
We propose a novel personality detection method, called PsyCoT, which mimics the way individuals complete psychological questionnaires in a multi-turn dialogue manner.
Our experiments demonstrate that PsyCoT significantly improves the performance and robustness of GPT-3.5 in personality detection.
arXiv Detail & Related papers (2023-10-31T08:23:33Z) - Editing Personality for Large Language Models [73.59001811199823]
This paper introduces an innovative task focused on editing the personality traits of Large Language Models (LLMs)
We construct PersonalityEdit, a new benchmark dataset to address this task.
arXiv Detail & Related papers (2023-10-03T16:02:36Z) - Preserving Knowledge Invariance: Rethinking Robustness Evaluation of
Open Information Extraction [50.62245481416744]
We present the first benchmark that simulates the evaluation of open information extraction models in the real world.
We design and annotate a large-scale testbed in which each example is a knowledge-invariant clique.
By further elaborating the robustness metric, a model is judged to be robust if its performance is consistently accurate on the overall cliques.
arXiv Detail & Related papers (2023-05-23T12:05:09Z) - Analysing the Impact of Audio Quality on the Use of Naturalistic
Long-Form Recordings for Infant-Directed Speech Research [62.997667081978825]
Modelling of early language acquisition aims to understand how infants bootstrap their language skills.
Recent developments have enabled the use of more naturalistic training data for computational models.
It is currently unclear how the sound quality could affect analyses and modelling experiments conducted on such data.
arXiv Detail & Related papers (2023-05-03T08:25:37Z) - Personality-aware Human-centric Multimodal Reasoning: A New Task,
Dataset and Baselines [32.82738983843281]
We introduce a new task called Personality-aware Human-centric Multimodal Reasoning (PHMR) (T1)
The goal of the task is to forecast the future behavior of a particular individual using multimodal information from past instances, while integrating personality factors.
The experimental results demonstrate that incorporating personality traits enhances human-centric multimodal reasoning performance.
arXiv Detail & Related papers (2023-04-05T09:09:10Z) - Domain-specific Learning of Multi-scale Facial Dynamics for Apparent
Personality Traits Prediction [3.19935268158731]
We propose a novel video-based automatic personality traits recognition approach.
It consists of: (1) a textbfdomain-specific facial behavior modelling module that extracts personality-related multi-scale short-term human facial behavior features; (2) a textbflong-term behavior modelling module that summarizes all short-term features of a video as a long-term/video-level personality representation; and (3) a textbfmulti-task personality traits prediction module that models underlying relationship among all traits and jointly predict them based on the video-level personality representation.
arXiv Detail & Related papers (2022-09-09T07:08:55Z) - Fact sheet: Automatic Self-Reported Personality Recognition Track [0.6091702876917281]
We analysed the correlation between the given metadata and the self-assigned personality trait scores.
We compared the performance of this informed baseline with models based on state-of-the-art visual, linguistic and audio features.
arXiv Detail & Related papers (2022-07-22T11:30:11Z) - Self-supervised models of audio effectively explain human cortical
responses to speech [71.57870452667369]
We capitalize on the progress of self-supervised speech representation learning to create new state-of-the-art models of the human auditory system.
We show that these results show that self-supervised models effectively capture the hierarchy of information relevant to different stages of speech processing in human cortex.
arXiv Detail & Related papers (2022-05-27T22:04:02Z) - Multitask Learning for Emotion and Personality Detection [17.029426018676997]
We build on the known correlation between personality traits and emotional behaviors, and propose a novel multitask learning framework, SoGMTL.
Our more computationally efficient CNN-based multitask model achieves the state-of-the-art performance across multiple famous personality and emotion datasets.
arXiv Detail & Related papers (2021-01-07T03:09:55Z) - Personality Trait Detection Using Bagged SVM over BERT Word Embedding
Ensembles [10.425280599592865]
We present a novel deep learning-based approach for automated personality detection from text.
We leverage state of the art advances in natural language understanding, namely the BERT language model to extract contextualized word embeddings.
Our model outperforms the previous state of the art by 1.04% and, at the same time is significantly more computationally efficient to train.
arXiv Detail & Related papers (2020-10-03T09:25:51Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.