Exploring the Impact of Occupational Personas on Domain-Specific QA

• URL: http://arxiv.org/abs/2505.24448v1
• Date: Fri, 30 May 2025 10:35:39 GMT
• Title: Exploring the Impact of Occupational Personas on Domain-Specific QA
• Authors: Eojin Kang, Jaehyuk Yu, Juae Kim,
• Abstract summary: This study analyzes whether personas enhance specialized QA performance by introducing two types of persona: Profession-Based Personas (PBPs) and Occupational Personality-Based Personas (OPBPs)<n>We demonstrate that while PBPs can slightly improve accuracy, OPBPs often degrade performance, even when semantically related to the task.<n>Our findings suggest that persona relevance alone does not guarantee effective knowledge utilization and that they may impose cognitive constraints that hinder optimal knowledge application.
• Score: 3.3590922002216193
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent studies on personas have improved the way Large Language Models (LLMs) interact with users. However, the effect of personas on domain-specific question-answering (QA) tasks remains a subject of debate. This study analyzes whether personas enhance specialized QA performance by introducing two types of persona: Profession-Based Personas (PBPs) (e.g., scientist), which directly relate to domain expertise, and Occupational Personality-Based Personas (OPBPs) (e.g., scientific person), which reflect cognitive tendencies rather than explicit expertise. Through empirical evaluations across multiple scientific domains, we demonstrate that while PBPs can slightly improve accuracy, OPBPs often degrade performance, even when semantically related to the task. Our findings suggest that persona relevance alone does not guarantee effective knowledge utilization and that they may impose cognitive constraints that hinder optimal knowledge application. Future research can explore how nuanced distinctions in persona representations guide LLMs, potentially contributing to reasoning and knowledge retrieval that more closely mirror human social conceptualization.

Related papers

• What Makes a Good Natural Language Prompt? [72.3282960118995]
  We conduct a meta-analysis surveying more than 150 prompting-related papers from leading NLP and AI conferences from 2022 to 2025.<n>We propose a property- and human-centric framework for evaluating prompt quality, encompassing 21 properties categorized into six dimensions.<n>We then empirically explore multi-property prompt enhancements in reasoning tasks, observing that single-property enhancements often have the greatest impact.
  arXiv  Detail & Related papers  (2025-06-07T23:19:27Z)
• Two Experts Are All You Need for Steering Thinking: Reinforcing Cognitive Effort in MoE Reasoning Models Without Additional Training [86.70255651945602]
  We introduce a novel inference-time steering methodology called Reinforcing Cognitive Experts (RICE)<n>RICE aims to improve reasoning performance without additional training or complexs.<n> Empirical evaluations with leading MoE-based LRMs demonstrate noticeable and consistent improvements in reasoning accuracy, cognitive efficiency, and cross-domain generalization.
  arXiv  Detail & Related papers  (2025-05-20T17:59:16Z)
• The potential -- and the pitfalls -- of using pre-trained language models as cognitive science theories [2.6549754445378344]
  We discuss challenges to the use of PLMs as cognitive science theories.<n>We review assumptions used by researchers to map measures of PLM performance to measures of human performance.<n>We end by enumerating criteria for using PLMs as credible accounts of cognition and cognitive development.
  arXiv  Detail & Related papers  (2025-01-22T05:24:23Z)
• PersonaFlow: Boosting Research Ideation with LLM-Simulated Expert Personas [12.593617990325528]
  We introduce PersonaFlow, an LLM-based system using persona simulation to support research ideation. Our findings indicate that using multiple personas during ideation significantly enhances user-perceived quality of outcomes. Users' persona customization interactions significantly improved their sense of control and recall of generated ideas.
  arXiv  Detail & Related papers  (2024-09-19T07:54:29Z)
• Building Better AI Agents: A Provocation on the Utilisation of Persona in LLM-based Conversational Agents [4.8916211213796394]
  This paper begins by examining the rationale and implications of imbuing CAs with unique personas. We delve into the specific applications where the implementation of a persona is not just beneficial but critical for LLM-based CAs. The paper underscores the necessity of a nuanced approach to persona integration, highlighting the potential challenges and ethical dilemmas that may arise.
  arXiv  Detail & Related papers  (2024-05-26T11:36:48Z)
• Prompting Explicit and Implicit Knowledge for Multi-hop Question Answering Based on Human Reading Process [6.394137489788181]
  Psychological studies suggest a vital connection between explicit information in passages and human prior knowledge during reading. We introduce a Prompting Explicit and Implicit knowledge (PEI) framework, which uses prompts to connect explicit and implicit knowledge. Our model incorporates type-specific reasoning via prompts, a form of implicit knowledge.
  arXiv  Detail & Related papers  (2024-02-29T16:56:36Z)
• PsyCoT: Psychological Questionnaire as Powerful Chain-of-Thought for Personality Detection [50.66968526809069]
  We propose a novel personality detection method, called PsyCoT, which mimics the way individuals complete psychological questionnaires in a multi-turn dialogue manner. Our experiments demonstrate that PsyCoT significantly improves the performance and robustness of GPT-3.5 in personality detection.
  arXiv  Detail & Related papers  (2023-10-31T08:23:33Z)
• From Heuristic to Analytic: Cognitively Motivated Strategies for Coherent Physical Commonsense Reasoning [66.98861219674039]
  Heuristic-Analytic Reasoning (HAR) strategies drastically improve the coherence of rationalizations for model decisions. Our findings suggest that human-like reasoning strategies can effectively improve the coherence and reliability of PLM reasoning.
  arXiv  Detail & Related papers  (2023-10-24T19:46:04Z)
• Revisiting the Reliability of Psychological Scales on Large Language Models [62.57981196992073]
  This study aims to determine the reliability of applying personality assessments to Large Language Models. Analysis of 2,500 settings per model, including GPT-3.5, GPT-4, Gemini-Pro, and LLaMA-3.1, reveals that various LLMs show consistency in responses to the Big Five Inventory.
  arXiv  Detail & Related papers  (2023-05-31T15:03:28Z)
• Interactive Natural Language Processing [67.87925315773924]
  Interactive Natural Language Processing (iNLP) has emerged as a novel paradigm within the field of NLP. This paper offers a comprehensive survey of iNLP, starting by proposing a unified definition and framework of the concept.
  arXiv  Detail & Related papers  (2023-05-22T17:18:29Z)
• Assessing Human Interaction in Virtual Reality With Continually Learning Prediction Agents Based on Reinforcement Learning Algorithms: A Pilot Study [6.076137037890219]
  We investigate how the interaction between a human and a continually learning prediction agent develops as the agent develops competency. We develop a virtual reality environment and a time-based prediction task wherein learned predictions from a reinforcement learning (RL) algorithm augment human predictions. Our findings suggest that human trust of the system may be influenced by early interactions with the agent, and that trust in turn affects strategic behaviour.
  arXiv  Detail & Related papers  (2021-12-14T22:46:44Z)
• You Impress Me: Dialogue Generation via Mutual Persona Perception [62.89449096369027]
  The research in cognitive science suggests that understanding is an essential signal for a high-quality chit-chat conversation. Motivated by this, we propose P2 Bot, a transmitter-receiver based framework with the aim of explicitly modeling understanding.
  arXiv  Detail & Related papers  (2020-04-11T12:51:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.