Constructing a Testbed for Psychometric Natural Language Processing
- URL: http://arxiv.org/abs/2007.12969v1
- Date: Sat, 25 Jul 2020 16:29:24 GMT
- Title: Constructing a Testbed for Psychometric Natural Language Processing
- Authors: Ahmed Abbasi, David G. Dobolyi, Richard G. Netemeyer
- Abstract summary: We describe our efforts to construct a corpus for psychometric natural language processing (NLP)
We discuss our multi-step process to align user text with their survey-based response items.
We report preliminary results on the use of the text to categorize/predict users' survey response labels.
- Score: 0.5801044612920815
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Psychometric measures of ability, attitudes, perceptions, and beliefs are
crucial for understanding user behaviors in various contexts including health,
security, e-commerce, and finance. Traditionally, psychometric dimensions have
been measured and collected using survey-based methods. Inferring such
constructs from user-generated text could afford opportunities for timely,
unobtrusive, collection and analysis. In this paper, we describe our efforts to
construct a corpus for psychometric natural language processing (NLP). We
discuss our multi-step process to align user text with their survey-based
response items and provide an overview of the resulting testbed which
encompasses survey-based psychometric measures and accompanying user-generated
text from over 8,500 respondents. We report preliminary results on the use of
the text to categorize/predict users' survey response labels. We also discuss
the important implications of our work and resulting testbed for future
psychometric NLP research.
Related papers
- LLM Questionnaire Completion for Automatic Psychiatric Assessment [49.1574468325115]
We employ a Large Language Model (LLM) to convert unstructured psychological interviews into structured questionnaires spanning various psychiatric and personality domains.
The obtained answers are coded as features, which are used to predict standardized psychiatric measures of depression (PHQ-8) and PTSD (PCL-C)
arXiv Detail & Related papers (2024-06-09T09:03:11Z) - Estimating Lexical Complexity from Document-Level Distributions [0.0]
We develop a two-step approach for estimating lexical complexity that does not rely on any pre-annotated data.
We also investigate the relationship between our complexity measure and certain features typically associated with complexity in the literature.
arXiv Detail & Related papers (2024-04-01T15:55:18Z) - PsyCoT: Psychological Questionnaire as Powerful Chain-of-Thought for
Personality Detection [50.66968526809069]
We propose a novel personality detection method, called PsyCoT, which mimics the way individuals complete psychological questionnaires in a multi-turn dialogue manner.
Our experiments demonstrate that PsyCoT significantly improves the performance and robustness of GPT-3.5 in personality detection.
arXiv Detail & Related papers (2023-10-31T08:23:33Z) - Concepts and Experiments on Psychoanalysis Driven Computing [0.0]
This research investigates the effective incorporation of the human factor and user perception in text-based interactive media.
We use the notion of Lacanian discourse types to capture and deeply understand real characteristics, qualities and contents of texts.
This is the first time computational methods are systematically combined with psychoanalysis.
arXiv Detail & Related papers (2022-09-29T19:27:22Z) - Predicting Human Psychometric Properties Using Computational Language
Models [5.806723407090421]
Transformer-based language models (LMs) continue to achieve state-of-the-art performance on natural language processing (NLP) benchmarks.
Can LMs be of use in predicting the psychometric properties of test items, when those items are given to human participants?
We gather responses from numerous human participants and LMs on a broad diagnostic test of linguistic competencies.
We then use the human responses to calculate standard psychometric properties of the items in the diagnostic test, using the human responses and the LM responses separately.
arXiv Detail & Related papers (2022-05-12T16:40:12Z) - Automated Quality Assessment of Cognitive Behavioral Therapy Sessions
Through Highly Contextualized Language Representations [34.670548892766625]
A BERT-based model is proposed for automatic behavioral scoring of a specific type of psychotherapy, called Cognitive Behavioral Therapy (CBT)
The model is trained in a multi-task manner in order to achieve higher interpretability.
BERT-based representations are further augmented with available therapy metadata, providing relevant non-linguistic context and leading to consistent performance improvements.
arXiv Detail & Related papers (2021-02-23T09:22:29Z) - Pose-based Body Language Recognition for Emotion and Psychiatric Symptom
Interpretation [75.3147962600095]
We propose an automated framework for body language based emotion recognition starting from regular RGB videos.
In collaboration with psychologists, we extend the framework for psychiatric symptom prediction.
Because a specific application domain of the proposed framework may only supply a limited amount of data, the framework is designed to work on a small training set.
arXiv Detail & Related papers (2020-10-30T18:45:16Z) - Vyaktitv: A Multimodal Peer-to-Peer Hindi Conversations based Dataset
for Personality Assessment [50.15466026089435]
We present a novel peer-to-peer Hindi conversation dataset- Vyaktitv.
It consists of high-quality audio and video recordings of the participants, with Hinglish textual transcriptions for each conversation.
The dataset also contains a rich set of socio-demographic features, like income, cultural orientation, amongst several others, for all the participants.
arXiv Detail & Related papers (2020-08-31T17:44:28Z) - How are you? Introducing stress-based text tailoring [63.128912221732946]
We discuss customising texts based on user stress level, as it could represent a critical factor when it comes to user engagement and behavioural change.
We first show a real-world example in which user behaviour is influenced by stress, then, after discussing which tools can be employed to assess and measure it, we propose an initial method for tailoring the document.
arXiv Detail & Related papers (2020-07-20T09:43:11Z) - A Revised Generative Evaluation of Visual Dialogue [80.17353102854405]
We propose a revised evaluation scheme for the VisDial dataset.
We measure consensus between answers generated by the model and a set of relevant answers.
We release these sets and code for the revised evaluation scheme as DenseVisDial.
arXiv Detail & Related papers (2020-04-20T13:26:45Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.