Individual Turing Test: A Case Study of LLM-based Simulation Using Longitudinal Personal Data

• URL: http://arxiv.org/abs/2603.01289v1
• Date: Sun, 01 Mar 2026 21:46:27 GMT
• Title: Individual Turing Test: A Case Study of LLM-based Simulation Using Longitudinal Personal Data
• Authors: Minghao Guo, Ziyi Ye, Wujiang Xu, Xi Zhu, Wenyue Hua, Dimitris N. Metaxas,
• Abstract summary: Large Language Models (LLMs) have demonstrated remarkable human-like capabilities, yet their ability to replicate a specific individual remains under-explored.<n>This paper presents a case study to investigate LLM-based individual simulation with a volunteer-contributed archive of private messaging history spanning over ten years.<n>We propose the "Individual Turing Test" to evaluate whether acquaintances of the volunteer can correctly identify which response in a multi-candidate pool most plausibly comes from the volunteer.
• Score: 54.145424717168794
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models (LLMs) have demonstrated remarkable human-like capabilities, yet their ability to replicate a specific individual remains under-explored. This paper presents a case study to investigate LLM-based individual simulation with a volunteer-contributed archive of private messaging history spanning over ten years. Based on the messaging data, we propose the "Individual Turing Test" to evaluate whether acquaintances of the volunteer can correctly identify which response in a multi-candidate pool most plausibly comes from the volunteer. We investigate prevalent LLM-based individual simulation approaches including: fine-tuning, retrieval-augmented generation (RAG), memory-based approach, and hybrid methods that integrate fine-tuning and RAG or memory. Empirical results show that current LLM-based simulation methods do not pass the Individual Turing Test, but they perform substantially better when the same test is conducted on strangers to the target individual. Additionally, while fine-tuning improves the simulation in daily chats representing the language style of the individual, retrieval-augmented and memory-based approaches demonstrate stronger performance on questions involving personal opinions and preferences. These findings reveal a fundamental trade-off between parametric and non-parametric approaches to individual simulation with LLMs when given a longitudinal context.

Related papers

• HumanStudy-Bench: Towards AI Agent Design for Participant Simulation [11.906370453952265]
  Large language models (LLMs) are increasingly used as simulated participants in social science experiments.<n>We introduce HUMANSTUDY-BENCH, a benchmark and execution engine that orchestrates LLM-based agents to reconstruct human-subject experiments.<n>To evaluate fidelity at the level of scientific inference, we propose new metrics to quantify how much human and agent behaviors agree.
  arXiv  Detail & Related papers  (2026-01-31T12:07:42Z)
• Computational Turing Test Reveals Systematic Differences Between Human and AI Language [0.0]
  Large language models (LLMs) are increasingly used in the social sciences to simulate human behavior.<n>Existing validation efforts rely heavily on human-judgment-based evaluations.<n>This paper introduces a computational Turing test to assess how closely LLMs approximate human language.
  arXiv  Detail & Related papers  (2025-11-06T08:56:37Z)
• SimBench: Benchmarking the Ability of Large Language Models to Simulate Human Behaviors [58.87134689752605]
  We introduce SimBench, the first large-scale, standardized benchmark for a robust, reproducible science of LLM simulation.<n>We show that even the best LLMs today have limited simulation ability (score: 40.80/100), performance scales log-linearly with model size.<n>We demonstrate that simulation ability correlates most strongly with deep, knowledge-intensive reasoning.
  arXiv  Detail & Related papers  (2025-10-20T13:14:38Z)
• Large Language Models as Virtual Survey Respondents: Evaluating Sociodemographic Response Generation [18.225151370273093]
  This paper explores a new paradigm: simulating virtual survey respondents using Large Language Models (LLMs)<n>We introduce two novel simulation settings, namely Partial Attribute Simulation (PAS) and Full Attribute Simulation (FAS)<n>We curate a comprehensive benchmark suite, LLM-S3 (Large Language Model-based Sociodemographic Simulation Survey), that spans 11 real-world public datasets across four sociological domains.
  arXiv  Detail & Related papers  (2025-09-08T04:59:00Z)
• IROTE: Human-like Traits Elicitation of Large Language Model via In-Context Self-Reflective Optimization [66.6349183886101]
  We propose IROTE, a novel in-context method for stable and transferable trait elicitation.<n>We show that one single IROTE-generated self-reflection can induce LLMs' stable impersonation of the target trait across diverse downstream tasks.
  arXiv  Detail & Related papers  (2025-08-12T08:04:28Z)
• IDA-Bench: Evaluating LLMs on Interactive Guided Data Analysis [60.32962597618861]
  IDA-Bench is a novel benchmark evaluating large language models in multi-round interactive scenarios.<n>Agent performance is judged by comparing its final numerical output to the human-derived baseline.<n>Even state-of-the-art coding agents (like Claude-3.7-thinking) succeed on 50% of the tasks, highlighting limitations not evident in single-turn tests.
  arXiv  Detail & Related papers  (2025-05-23T09:37:52Z)
• Deep Binding of Language Model Virtual Personas: a Study on Approximating Political Partisan Misperceptions [13.929713456538932]
  We propose a novel methodology for constructing virtual personas with synthetic user "backstories" generated as extended, multi-turn interview transcripts.<n>We show that virtual personas conditioned on our backstories closely replicate human response distributions and produce effect sizes that closely match those observed in the original studies of in-group/out-group biases.
  arXiv  Detail & Related papers  (2025-04-16T00:10:34Z)
• Mixture-of-Personas Language Models for Population Simulation [20.644911871150136]
  Large Language Models (LLMs) can augment human-generated data in social science research and machine learning model training.<n>MoP is a contextual mixture model, where each component is an LM agent characterized by a persona and an exemplar representing subpopulation behaviors.<n>MoP is flexible, requires no model finetuning, and is transferable across base models.
  arXiv  Detail & Related papers  (2025-04-07T12:43:05Z)
• If an LLM Were a Character, Would It Know Its Own Story? Evaluating Lifelong Learning in LLMs [55.8331366739144]
  We introduce LIFESTATE-BENCH, a benchmark designed to assess lifelong learning in large language models (LLMs)<n>Our fact checking evaluation probes models' self-awareness, episodic memory retrieval, and relationship tracking, across both parametric and non-parametric approaches.
  arXiv  Detail & Related papers  (2025-03-30T16:50:57Z)
• LLM-Mirror: A Generated-Persona Approach for Survey Pre-Testing [0.0]
  We investigate whether providing respondents' prior information can replicate both statistical distributions and individual decision-making patterns.<n>We also introduce the concept of the LLM-Mirror, user personas generated by supplying respondent-specific information to the LLM.<n>Our findings show that: (1) PLS-SEM analysis shows LLM-generated responses align with human responses, (2) LLMs are capable of reproducing individual human responses, and (3) LLM-Mirror responses closely follow human responses at the individual level.
  arXiv  Detail & Related papers  (2024-12-04T09:39:56Z)
• MemSim: A Bayesian Simulator for Evaluating Memory of LLM-based Personal Assistants [64.41695570145673]
  We propose MemSim, a Bayesian simulator designed to automatically construct reliable questions and answers (QAs) from generated user messages. Based on MemSim, we generate a dataset in the daily-life scenario, named MemDaily, and conduct extensive experiments to assess the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-09-30T10:19:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.