When Can Digital Personas Reliably Approximate Human Survey Findings?

• URL: http://arxiv.org/abs/2605.10659v1
• Date: Mon, 11 May 2026 14:41:11 GMT
• Title: When Can Digital Personas Reliably Approximate Human Survey Findings?
• Abstract summary: Digital personas powered by Large Language Models (LLMs) are increasingly proposed as substitutes for human survey respondents.<n>It remains unclear when they can reliably approximate human survey findings.<n>We construct personas from respondents' background variables and pre-2023 survey histories, then testing them against the same respondents' held-out post-cutoff answers.
• Score: 0.8411771830145969
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Digital personas powered by Large Language Models (LLMs) are increasingly proposed as substitutes for human survey respondents, yet it remains unclear when they can reliably approximate human survey findings. We answer this question using the LISS panel, constructing personas from respondents' background variables and pre-2023 survey histories, then testing them against the same respondents' held-out post-cutoff answers. Across four persona architectures, three LLMs, and two prediction tasks, we assess performance at the question, respondent, distributional, equity, and clustering levels. Digital personas improve alignment with human response distributions, especially in domains tied to stable attributes and values, but remain limited for individual prediction and fail to recover multivariate respondent structure. Retrieval-augmented architectures provide the clearest gains, but performance depends more on human response structure than on model choice: personas perform best for low-variability questions and common respondent patterns, and worst for subjective, heterogeneous, or rare responses. Our results provide practical guidance on when digital personas could be appropriate for survey research and when human validation remains necessary.

Related papers

• Individual Text Corpora Predict User-Specific Knowledge: Benchmarks of Individualized Knowledge Simulation [37.90336748891877]
  This study examines whether individual text corpora (ICs) from search histories can be used to simulate individual knowledge.<n>We collected ICs from 316 adults, who answered 36 multiple-choice knowledge items, and compared several large language models (LLMs) on this task.<n>After task-specific fine-tuning via Low-Rank Adaptation (LoRA), Qwen3-1.7B outperformed both participants and a representative German norm sample on publicly available items.
  arXiv  Detail & Related papers  (2026-09-08T10:19:28Z)
• Rectification Difficulty and Optimal Sample Allocation in LLM-Augmented Surveys [0.0]
  Large Language Models can generate synthetic survey responses at low cost, but their accuracy varies unpredictably across questions.<n>We study the design problem of allocating a fixed budget of human respondents across estimation tasks when cheap LLM predictions are available for every task.<n>Our framework combines three components. First, building on Prediction-Powered Inference, we characterize a question-specific rectification difficulty that governs how quickly the estimator's variance decreases with human sample size.<n>Second, we derive a closed-form optimal allocation rule that directs more human labels to tasks where the LLM is least reliable.
  arXiv  Detail & Related papers  (2026-04-19T05:45:37Z)
• Assessing the Reliability of Persona-Conditioned LLMs as Synthetic Survey Respondents [0.4277616907160855]
  We use a large dataset of U.S. microdata to assess the impact of persona-conditioned simulations.<n>We find that persona prompting does not yield a clear aggregate improvement in survey alignment and, in many cases, significantly degrades performance.<n>Our findings highlight a key adverse impact of current persona-based simulation practices.
  arXiv  Detail & Related papers  (2026-02-06T15:13:59Z)
• Pathways of Thoughts: Multi-Directional Thinking for Long-form Personalized Question Answering [57.12316804290369]
  Personalization is essential for adapting question answering systems to user-specific information needs.<n>We propose Pathways of Thoughts (PoT), an inference-stage method that applies to any large language model (LLM) without requiring task-specific fine-tuning.<n>PoT consistently outperforms competitive baselines, achieving up to a 13.1% relative improvement.
  arXiv  Detail & Related papers  (2025-09-23T14:44:46Z)
• HumanPCR: Probing MLLM Capabilities in Diverse Human-Centric Scenes [72.26829188852139]
  HumanPCR is an evaluation suite for probing MLLMs' capacity about human-related visual contexts.<n>Human-P, HumanThought-C, and Human-R feature over 6,000 human-verified multiple choice questions.<n>Human-R offers a challenging manually curated video reasoning test.
  arXiv  Detail & Related papers  (2025-08-19T09:52:04Z)
• Prompt Perturbations Reveal Human-Like Biases in Large Language Model Survey Responses [2.3112192919085826]
  Large Language Models (LLMs) are increasingly used as proxies for human subjects in social science surveys.<n>Their reliability and susceptibility to known human-like response biases are poorly understood.<n>This work investigates the response robustness of LLMs in normative survey contexts.
  arXiv  Detail & Related papers  (2025-07-09T18:01:50Z)
• Cascaded Language Models for Cost-effective Human-AI Decision-Making [52.81324217423194]
  We present a cascaded LLM decision framework that adaptively delegates tasks across multiple tiers of expertise.<n>First, a deferral policy determines whether to accept the base model's answer or regenerate it with a large model.<n>Second, an abstention policy decides whether the cascade model response is sufficiently certain or requires human intervention.
  arXiv  Detail & Related papers  (2025-06-13T15:36:22Z)
• Leveraging Interview-Informed LLMs to Model Survey Responses: Comparative Insights from AI-Generated and Human Data [4.774576759157642]
  Mixed methods research integrates quantitative and qualitative data but faces challenges in aligning their distinct structures.<n>This study investigates whether large language models (LLMs) can reliably predict human survey responses.
  arXiv  Detail & Related papers  (2025-05-28T05:57:26Z)
• Specializing Large Language Models to Simulate Survey Response Distributions for Global Populations [49.908708778200115]
  We are the first to specialize large language models (LLMs) for simulating survey response distributions.<n>As a testbed, we use country-level results from two global cultural surveys.<n>We devise a fine-tuning method based on first-token probabilities to minimize divergence between predicted and actual response distributions.
  arXiv  Detail & Related papers  (2025-02-10T21:59:27Z)
• LLM-Mirror: A Generated-Persona Approach for Survey Pre-Testing [0.0]
  We investigate whether providing respondents' prior information can replicate both statistical distributions and individual decision-making patterns.<n>We also introduce the concept of the LLM-Mirror, user personas generated by supplying respondent-specific information to the LLM.<n>Our findings show that: (1) PLS-SEM analysis shows LLM-generated responses align with human responses, (2) LLMs are capable of reproducing individual human responses, and (3) LLM-Mirror responses closely follow human responses at the individual level.
  arXiv  Detail & Related papers  (2024-12-04T09:39:56Z)
• Quriosity: Analyzing Human Questioning Behavior and Causal Inquiry through Curiosity-Driven Queries [92.1651731484397]
  We present Quriosity, a collection of 13.5K naturally occurring questions from three diverse sources.<n>Our analysis reveals a significant presence of causal questions (up to 42%) in the dataset.
  arXiv  Detail & Related papers  (2024-05-30T17:55:28Z)
• Do LLMs exhibit human-like response biases? A case study in survey design [66.1850490474361]
  We investigate the extent to which large language models (LLMs) reflect human response biases, if at all. We design a dataset and framework to evaluate whether LLMs exhibit human-like response biases in survey questionnaires. Our comprehensive evaluation of nine models shows that popular open and commercial LLMs generally fail to reflect human-like behavior.
  arXiv  Detail & Related papers  (2023-11-07T15:40:43Z)
• ExpertQA: Expert-Curated Questions and Attributed Answers [51.68314045809179]
  We conduct human evaluation of responses from a few representative systems along various axes of attribution and factuality. We collect expert-curated questions from 484 participants across 32 fields of study, and then ask the same experts to evaluate generated responses to their own questions. The output of our analysis is ExpertQA, a high-quality long-form QA dataset with 2177 questions spanning 32 fields, along with verified answers and attributions for claims in the answers.
  arXiv  Detail & Related papers  (2023-09-14T16:54:34Z)
• On the Efficacy of Adversarial Data Collection for Question Answering: Results from a Large-Scale Randomized Study [65.17429512679695]
  In adversarial data collection (ADC), a human workforce interacts with a model in real time, attempting to produce examples that elicit incorrect predictions. Despite ADC's intuitive appeal, it remains unclear when training on adversarial datasets produces more robust models.
  arXiv  Detail & Related papers  (2021-06-02T00:48:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.