Personality-aware Human-centric Multimodal Reasoning: A New Task,
Dataset and Baselines
- URL: http://arxiv.org/abs/2304.02313v2
- Date: Mon, 4 Mar 2024 12:33:46 GMT
- Title: Personality-aware Human-centric Multimodal Reasoning: A New Task,
Dataset and Baselines
- Authors: Yaochen Zhu, Xiangqing Shen, Rui Xia
- Abstract summary: We introduce a new task called Personality-aware Human-centric Multimodal Reasoning (PHMR) (T1)
The goal of the task is to forecast the future behavior of a particular individual using multimodal information from past instances, while integrating personality factors.
The experimental results demonstrate that incorporating personality traits enhances human-centric multimodal reasoning performance.
- Score: 32.82738983843281
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Personality traits, emotions, and beliefs shape individuals' behavioral
choices and decision-making processes. However, for one thing, the affective
computing community normally focused on predicting personality traits but
overlooks their application in behavior prediction. For another, the multimodal
reasoning task emphasized the prediction of future states and behaviors but
often neglected the incorporation of individual personality traits. In this
work, we introduce a new task called Personality-aware Human-centric Multimodal
Reasoning (PHMR) (T1), with the goal of forecasting the future behavior of a
particular individual using multimodal information from past instances, while
integrating personality factors. We accordingly construct a new dataset based
on six television shows, encompassing 225 characters and 12k samples. To
establish a benchmark for the task, we propose seven baseline methods: three
adapted from related tasks, two pre-trained model, and two multimodal large
language models. The experimental results demonstrate that incorporating
personality traits enhances human-centric multimodal reasoning performance. To
further solve the lack of personality annotation in real-life scenes, we
introduce an extension task called Personality-predicted Human-centric
Multimodal Reasoning task (T2) along with the corresponding dataset and method.
We will make our dataset and code available on GitHub.
Related papers
- Multi-Transmotion: Pre-trained Model for Human Motion Prediction [68.87010221355223]
Multi-Transmotion is an innovative transformer-based model designed for cross-modality pre-training.
Our methodology demonstrates competitive performance across various datasets on several downstream tasks.
arXiv Detail & Related papers (2024-11-04T23:15:21Z) - PersonalityScanner: Exploring the Validity of Personality Assessment Based on Multimodal Signals in Virtual Reality [44.15145632980038]
PersonalityScanner is a VR simulator to stimulate cognitive processes and simulate daily behaviors.
We collect a synchronous multi-modal dataset with ten modalities, including first/third-person video, audio, text, eye tracking, facial microexpression, pose, depth data, log, and inertial measurement unit.
arXiv Detail & Related papers (2024-07-29T06:17:41Z) - PsyCoT: Psychological Questionnaire as Powerful Chain-of-Thought for
Personality Detection [50.66968526809069]
We propose a novel personality detection method, called PsyCoT, which mimics the way individuals complete psychological questionnaires in a multi-turn dialogue manner.
Our experiments demonstrate that PsyCoT significantly improves the performance and robustness of GPT-3.5 in personality detection.
arXiv Detail & Related papers (2023-10-31T08:23:33Z) - Editing Personality for Large Language Models [73.59001811199823]
This paper introduces an innovative task focused on editing the personality traits of Large Language Models (LLMs)
We construct PersonalityEdit, a new benchmark dataset to address this task.
arXiv Detail & Related papers (2023-10-03T16:02:36Z) - Learning signatures of decision making from many individuals playing the
same game [54.33783158658077]
We design a predictive framework that learns representations to encode an individual's 'behavioral style'
We apply our method to a large-scale behavioral dataset from 1,000 humans playing a 3-armed bandit task.
arXiv Detail & Related papers (2023-02-21T21:41:53Z) - Domain-specific Learning of Multi-scale Facial Dynamics for Apparent
Personality Traits Prediction [3.19935268158731]
We propose a novel video-based automatic personality traits recognition approach.
It consists of: (1) a textbfdomain-specific facial behavior modelling module that extracts personality-related multi-scale short-term human facial behavior features; (2) a textbflong-term behavior modelling module that summarizes all short-term features of a video as a long-term/video-level personality representation; and (3) a textbfmulti-task personality traits prediction module that models underlying relationship among all traits and jointly predict them based on the video-level personality representation.
arXiv Detail & Related papers (2022-09-09T07:08:55Z) - Two-Faced Humans on Twitter and Facebook: Harvesting Social Multimedia
for Human Personality Profiling [74.83957286553924]
We infer the Myers-Briggs Personality Type indicators by applying a novel multi-view fusion framework, called "PERS"
Our experimental results demonstrate the PERS's ability to learn from multi-view data for personality profiling by efficiently leveraging on the significantly different data arriving from diverse social multimedia sources.
arXiv Detail & Related papers (2021-06-20T10:48:49Z) - Multitask Learning for Emotion and Personality Detection [17.029426018676997]
We build on the known correlation between personality traits and emotional behaviors, and propose a novel multitask learning framework, SoGMTL.
Our more computationally efficient CNN-based multitask model achieves the state-of-the-art performance across multiple famous personality and emotion datasets.
arXiv Detail & Related papers (2021-01-07T03:09:55Z) - Model-agnostic Fits for Understanding Information Seeking Patterns in
Humans [0.0]
In decision making tasks under uncertainty, humans display characteristic biases in seeking, integrating, and acting upon information relevant to the task.
Here, we reexamine data from previous carefully designed experiments, collected at scale, that measured and catalogued these biases in aggregate form.
We design deep learning models that replicate these biases in aggregate, while also capturing individual variation in behavior.
arXiv Detail & Related papers (2020-12-09T04:34:58Z) - Vyaktitv: A Multimodal Peer-to-Peer Hindi Conversations based Dataset
for Personality Assessment [50.15466026089435]
We present a novel peer-to-peer Hindi conversation dataset- Vyaktitv.
It consists of high-quality audio and video recordings of the participants, with Hinglish textual transcriptions for each conversation.
The dataset also contains a rich set of socio-demographic features, like income, cultural orientation, amongst several others, for all the participants.
arXiv Detail & Related papers (2020-08-31T17:44:28Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.