Synthetic Personalities: How Well Can LLMs Mimic Individual Respondents Using Socio-Economic Microdata?

• URL: http://arxiv.org/abs/2606.04592v1
• Date: Wed, 03 Jun 2026 08:30:03 GMT
• Title: Synthetic Personalities: How Well Can LLMs Mimic Individual Respondents Using Socio-Economic Microdata?
• Abstract summary: We construct detailed individual-level twins from the German Socio-Economic Panel.<n>We evaluate them across a $3 times 5 times 2 times 2$ construction-method grid.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: LLM-based digital twins promise to scale and accelerate market research, but most published twins are either coarse persona bots conditioned on a few demographic questions or detailed individual-level twins built on purpose-collected surveys and interview transcripts. Neither setup speaks to the operationally most relevant case for marketing practice: building detailed individual twins from the pre-existing heterogeneous panel data that firms already accumulate through CRM systems, loyalty programs, and repeat surveys. We construct detailed individual-level twins from the German Socio-Economic Panel (SOEP) and evaluate them across a $3 \times 5 \times 2 \times 2$ construction-method grid that covers three open-weights LLMs, five cumulative information depths ranked by normalized Shannon entropy, two embedding methods, and two reasoning modes, scoring over 2.1 million twin responses on 500 participants and 183 held-out questions. Twin quality rises with information depth but with diminishing returns past the 75 percent entropy quartile, which acts as a cost-efficient Pareto point relative to the best-performing 100 percent cells. Switching the embedding from a narrative persona summary to a raw dialog history of past responses raises hold-out accuracy in every model-by-reasoning cell at the 100 percent depth, while an explicit thinking mode raises rank-order correlation without moving accuracy. Best-cell accuracy reaches 78.8 percent and Fisher-$z$ correlation reaches $r = 0.590$ on the SOEP held-out evaluation set. The findings suggest that twin-based market research is no longer gated by data design, but by item volume, model selection, and a small set of construction-level decisions that this paper now maps.

Related papers

• Synthetic Data in Marketing Research: How to Evaluate and When to Trust [0.0]
  Debate over synthetic data in marketing research has polarized between claims that large language models (LLMs) make human respondents obsolete and calls to avoid them entirely.<n>We argue that both positions obscure the more useful question: not whether synthetic respondents work, but when.<n>First, we distinguish three types of synthetic data and map each to the decisions it can support.<n>Second, we develop a taxonomy of four families of accuracy measures and suggest that the wide range of reported twin accuracy, from near-perfect to near-chance, largely reflects differences in what is being measured rather than in method quality.
  arXiv  Detail & Related papers  (2026-09-12T15:13:10Z)
• AQuA: Recursively Self-Improving Quantitative Trading Research Agents [44.546401968461055]
  We study whether an autonomous system can use evidence from earlier experiments to improve the hypotheses and candidates proposed in later iterations.<n>We present AQuA, which comprises two separate language-model-driven research systems.
  arXiv  Detail & Related papers  (2026-08-13T05:25:42Z)
• Iterating Toward Better Search: A Two-Agent Simulation Framework for Evaluating Agentic Search Architectures in E-Commerce [5.346615622025758]
  We present a modular two-agent simulation framework for evaluating conversational shopping assistant architectures.<n> Holding the buyer constant across experiments enables controlled comparison of responder designs on identical scenarios.<n>Rolling-window memory outperforms intent-extraction memory on all quality metrics while being 35% faster per query.
  arXiv  Detail & Related papers  (2026-06-11T05:27:42Z)
• Persona Conditioning of Brand Recommendations in Retrieval-Augmented Commercial Chat: A Prominence-Stratified Cross-Provider Audit [0.0]
  We audit how strongly that contextual variation reshapes which brands the model recommends.<n>The audit samples 2,000 runs over a design space of 10 personas x 8 prompts x 3 model configurations x N=10 reps.<n>Any measurement of AI brand perception must condition on the buyer persona supplying the query.
  arXiv  Detail & Related papers  (2026-05-28T16:43:38Z)
• PersonalAI 2.0: Enhancing knowledge graph traversal/retrieval with planning mechanism for Personalized LLM Agents [53.369140518949656]
  We introduce PersonalAI 2.0 (PAI-2), a novel framework to enhance large language model (LLM) based systems through integration of external knowledge graphs (KG)<n>The proposed approach addresses key limitations of existing Retrieval-Augmented Generation (GraphRAG) methods by incorporating a dynamic, multistage query processing pipeline.<n>We show that use of graph algorithms (e.g. BeamSearch, WaterCircles) gain superior results compared to standard flatten retriever on average 6%, while enabled search plan enhancement mechanism gain 18% boost compared to disabled one by LLM-as-a-Judge across six datasets.
  arXiv  Detail & Related papers  (2026-05-13T13:06:30Z)
• Rectification Difficulty and Optimal Sample Allocation in LLM-Augmented Surveys [0.0]
  Large Language Models can generate synthetic survey responses at low cost, but their accuracy varies unpredictably across questions.<n>We study the design problem of allocating a fixed budget of human respondents across estimation tasks when cheap LLM predictions are available for every task.<n>Our framework combines three components. First, building on Prediction-Powered Inference, we characterize a question-specific rectification difficulty that governs how quickly the estimator's variance decreases with human sample size.<n>Second, we derive a closed-form optimal allocation rule that directs more human labels to tasks where the LLM is least reliable.
  arXiv  Detail & Related papers  (2026-04-19T05:45:37Z)
• RefineBench: Evaluating Refinement Capability of Language Models via Checklists [71.02281792867531]
  We evaluate two refinement modes: guided refinement and self-refinement.<n>In guided refinement, both proprietary LMs and large open-weight LMs can leverage targeted feedback to refine responses to near-perfect levels within five turns.<n>These findings suggest that frontier LMs require breakthroughs to self-refine their incorrect responses.
  arXiv  Detail & Related papers  (2025-11-27T07:20:52Z)
• Dynamic Template Selection for Output Token Generation Optimization: MLP-Based and Transformer Approaches [0.0]
  Dynamic template selection achieves significant cost reductions without compromising response quality.<n>This work contributes several key elements: formal problem formulation with theoretical grounding in machine learning, four algorithms with corresponding complexity analyses, and extensive empirical validation across production systems.
  arXiv  Detail & Related papers  (2025-11-17T21:00:22Z)
• Twin-2K-500: A dataset for building digital twins of over 2,000 people based on their answers to over 500 questions [11.751234495886674]
  LLM-based digital twin simulation holds great promise for research in AI, social science, and digital experimentation.<n>We survey a representative sample of $N = 2,058$ participants (average 2.42 hours per person) in the US across four waves with 500 questions in total.<n>Initial analyses suggest the data are of high quality and show promise for constructing digital twins that predict human behavior well at the individual and aggregate levels.
  arXiv  Detail & Related papers  (2025-05-23T05:05:11Z)
• Mind the Gap! Static and Interactive Evaluations of Large Audio Models [55.87220295533817]
  Large Audio Models (LAMs) are designed to power voice-native experiences.<n>This study introduces an interactive approach to evaluate LAMs and collect 7,500 LAM interactions from 484 participants.
  arXiv  Detail & Related papers  (2025-02-21T20:29:02Z)
• One Thousand and One Pairs: A "novel" challenge for long-context language models [56.60667988954638]
  NoCha is a dataset of 1,001 pairs of true and false claims about 67 fictional books. Our annotators confirm that the largest share of pairs in NoCha require global reasoning over the entire book to verify. On average, models perform much better on pairs that require only sentence-level retrieval vs. global reasoning.
  arXiv  Detail & Related papers  (2024-06-24T02:03:57Z)
• Long-Span Question-Answering: Automatic Question Generation and QA-System Ranking via Side-by-Side Evaluation [65.16137964758612]
  We explore the use of long-context capabilities in large language models to create synthetic reading comprehension data from entire books. Our objective is to test the capabilities of LLMs to analyze, understand, and reason over problems that require a detailed comprehension of long spans of text.
  arXiv  Detail & Related papers  (2024-05-31T20:15:10Z)
• Automatic sleep stage classification with deep residual networks in a mixed-cohort setting [63.52264764099532]
  We developed a novel deep neural network model to assess the generalizability of several large-scale cohorts. Overall classification accuracy improved with increasing fractions of training data.
  arXiv  Detail & Related papers  (2020-08-21T10:48:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.