Secret Keepers: The Impact of LLMs on Linguistic Markers of Personal Traits

• URL: http://arxiv.org/abs/2404.00267v2
• Date: Wed, 3 Apr 2024 17:29:12 GMT
• Title: Secret Keepers: The Impact of LLMs on Linguistic Markers of Personal Traits
• Authors: Zhivar Sourati, Meltem Ozcan, Colin McDaniel, Alireza Ziabari, Nuan Wen, Ala Tak, Fred Morstatter, Morteza Dehghani,
• Abstract summary: We investigate the impact of Large Language Models (LLMs) on the linguistic markers of demographic and psychological traits. Our findings indicate that although the use of LLMs slightly reduces the predictive power of linguistic patterns over authors' personal traits, the significant changes are infrequent.
• Score: 6.886654996060662
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Prior research has established associations between individuals' language usage and their personal traits; our linguistic patterns reveal information about our personalities, emotional states, and beliefs. However, with the increasing adoption of Large Language Models (LLMs) as writing assistants in everyday writing, a critical question emerges: are authors' linguistic patterns still predictive of their personal traits when LLMs are involved in the writing process? We investigate the impact of LLMs on the linguistic markers of demographic and psychological traits, specifically examining three LLMs - GPT3.5, Llama 2, and Gemini - across six different traits: gender, age, political affiliation, personality, empathy, and morality. Our findings indicate that although the use of LLMs slightly reduces the predictive power of linguistic patterns over authors' personal traits, the significant changes are infrequent, and the use of LLMs does not fully diminish the predictive power of authors' linguistic patterns over their personal traits. We also note that some theoretically established lexical-based linguistic markers lose their reliability as predictors when LLMs are used in the writing process. Our findings have important implications for the study of linguistic markers of personal traits in the age of LLMs.

Related papers

• The Dark Patterns of Personalized Persuasion in Large Language Models: Exposing Persuasive Linguistic Features for Big Five Personality Traits in LLMs Responses [0.0]
  We identify 13 linguistic features crucial for influencing personalities across different levels of the Big Five model of personality. Findings show that models use more anxiety-related words for neuroticism, increase achievement-related words for conscientiousness, and employ fewer cognitive processes words for openness to experience.
  arXiv  Detail & Related papers  (2024-11-08T23:02:59Z)
• Large Language Models Reflect the Ideology of their Creators [73.25935570218375]
  Large language models (LLMs) are trained on vast amounts of data to generate natural language. We uncover notable diversity in the ideological stance exhibited across different LLMs and languages.
  arXiv  Detail & Related papers  (2024-10-24T04:02:30Z)
• LMLPA: Language Model Linguistic Personality Assessment [11.599282127259736]
  Large Language Models (LLMs) are increasingly used in everyday life and research. measuring the personality of a given LLM is currently a challenge. This paper introduces the Language Model Linguistic Personality Assessment (LMLPA), a system designed to evaluate the linguistic personalities of LLMs.
  arXiv  Detail & Related papers  (2024-10-23T07:48:51Z)
• Hate Personified: Investigating the role of LLMs in content moderation [64.26243779985393]
  For subjective tasks such as hate detection, where people perceive hate differently, the Large Language Model's (LLM) ability to represent diverse groups is unclear. By including additional context in prompts, we analyze LLM's sensitivity to geographical priming, persona attributes, and numerical information to assess how well the needs of various groups are reflected.
  arXiv  Detail & Related papers  (2024-10-03T16:43:17Z)
• Inclusivity in Large Language Models: Personality Traits and Gender Bias in Scientific Abstracts [49.97673761305336]
  We evaluate three large language models (LLMs) for their alignment with human narrative styles and potential gender biases. Our findings indicate that, while these models generally produce text closely resembling human authored content, variations in stylistic features suggest significant gender biases.
  arXiv  Detail & Related papers  (2024-06-27T19:26:11Z)
• PhonologyBench: Evaluating Phonological Skills of Large Language Models [57.80997670335227]
  Phonology, the study of speech's structure and pronunciation rules, is a critical yet often overlooked component in Large Language Model (LLM) research. We present PhonologyBench, a novel benchmark consisting of three diagnostic tasks designed to explicitly test the phonological skills of LLMs. We observe a significant gap of 17% and 45% on Rhyme Word Generation and Syllable counting, respectively, when compared to humans.
  arXiv  Detail & Related papers  (2024-04-03T04:53:14Z)
• LLMvsSmall Model? Large Language Model Based Text Augmentation Enhanced Personality Detection Model [58.887561071010985]
  Personality detection aims to detect one's personality traits underlying in social media posts. Most existing methods learn post features directly by fine-tuning the pre-trained language models. We propose a large language model (LLM) based text augmentation enhanced personality detection model.
  arXiv  Detail & Related papers  (2024-03-12T12:10:18Z)
• Beware of Words: Evaluating the Lexical Diversity of Conversational LLMs using ChatGPT as Case Study [3.0059120458540383]
  We consider the evaluation of the lexical richness of the text generated by conversational Large Language Models (LLMs) and how it depends on the model parameters. The results show how lexical richness depends on the version of ChatGPT and some of its parameters, such as the presence penalty, or on the role assigned to the model.
  arXiv  Detail & Related papers  (2024-02-11T13:41:17Z)
• Revisiting the Reliability of Psychological Scales on Large Language Models [62.57981196992073]
  This study aims to determine the reliability of applying personality assessments to Large Language Models. Analysis of 2,500 settings per model, including GPT-3.5, GPT-4, Gemini-Pro, and LLaMA-3.1, reveals that various LLMs show consistency in responses to the Big Five Inventory.
  arXiv  Detail & Related papers  (2023-05-31T15:03:28Z)
• PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traits [30.770525830385637]
  We study the behavior of large language models (LLMs) based on the Big Five personality model. Results show that LLM personas' self-reported BFI scores are consistent with their designated personality types. Human evaluation shows that humans can perceive some personality traits with an accuracy of up to 80%.
  arXiv  Detail & Related papers  (2023-05-04T04:58:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.