When Persona Simulations Are Informative: Graph-Structured Signals for Pluralistic Opinion Sensing

• URL: http://arxiv.org/abs/2608.22438v1
• Date: Sun, 23 Aug 2026 14:33:35 GMT
• Title: When Persona Simulations Are Informative: Graph-Structured Signals for Pluralistic Opinion Sensing
• Abstract summary: Persona-Conditioned Informativeness ( PCI) is an unsupervised diagnostic metric.<n> PCI measures whether semantically similar personas deviate in concordant directions relative to item-level sample baselines.<n>These findings support PCI as a principled internal diagnostic for screening synthetic respondents in survey pipelines.
• Score: 0.6627152091494143
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Persona-conditioned large language models (LLMs) are increasingly used to simulate survey responses across diverse domains. However, apparent response variation can reflect unconditioned model priors or token sampling noise rather than systematic persona conditioning. We argue that persona-conditioned variation is informative when semantically similar personas exhibit concordant response shifts. To operationalize this principle, we introduce Persona-Conditioned Informativeness (PCI), an unsupervised diagnostic metric that measures whether semantically similar personas deviate in concordant directions relative to item-level sample baselines. By modeling personas as a similarity graph, PCI uses Local Moran's I to quantify local spatial coherence and extract compact persona subsets without using construct labels. To evaluate PCI without external human benchmarks, we test its ability to recover established latent value structure using the 57-item Portrait Values Questionnaire-Revised (PVQ-RR). Confirmatory factor analysis (CFA) shows that a PCI-selected 10% subset substantially improves overall construct recovery relative to response-stability and random selection. These findings support PCI as a principled internal diagnostic for screening synthetic respondents in survey pipelines.

Related papers

• Rethinking the Evaluation and Optimization of LLM-Based Social Simulation [56.18577997572273]
  We use the subjectivity coefficient to analyze how accuracy-based evaluation and hard-label training fail as subjectivity grows.<n>We construct SUBJSIM, a benchmark of 19,300 contexts covering 193 annotators and 100 subjective questions.<n>Results on SUBJSIM demonstrate the advantages of our method.
  arXiv  Detail & Related papers  (2026-08-20T06:26:03Z)
• When prompt perturbations break your A/B test: A valid statistical test for generative surveying [0.3366038232188622]
  We show that standard hypothesis tests, including the sign test and Wilcoxon signed-rank test, are invalid under a statistical model for generative surveying.<n>We propose a permutation test that is valid under this model and formally characterize the conditions under which standard tests fail.
  arXiv  Detail & Related papers  (2026-05-26T00:35:58Z)
• SAAG: Structured Agent Assessment and Grounding [13.62148061055744]
  We propose a cascaded diagnostic framework that decomposes agent-calling evaluation into three sequential stages.<n>We evaluate this framework on a benchmark derived from Glaive's function-calling dataset across registry sizes of 5, 10, and 15 agents.
  arXiv  Detail & Related papers  (2026-04-30T19:33:58Z)
• MCQ Difficulty Prediction via Modeling Learner Heterogeneity Using Data-Driven Cognitive Profiling [0.11458853556386796]
  We propose a persona-driven framework that replaces theoretical ability sampling with data-driven cognitive profiling.<n>We identify behavioral personas via latent class analysis (LCA), then condition a large language model (LLM) to simulate response distributions for each persona.<n>The discovered personas are interpretable and offer insights into why items are difficult, with potential applications to diagnostic assessment design.
  arXiv  Detail & Related papers  (2026-04-13T19:18:14Z)
• Interpretability and Individuality in Knee MRI: Patient-Specific Radiomic Fingerprint with Reconstructed Healthy Personas [40.168029561784216]
  A radiomic fingerprint is a patient-specific feature set derived from MRI.<n>A healthy persona synthesises a pathology-free baseline for each patient.<n>Comparing features extracted from pathological images against their personas highlights deviations from normal anatomy.
  arXiv  Detail & Related papers  (2026-01-13T14:48:01Z)
• Model Correlation Detection via Random Selection Probing [62.093777777813756]
  Existing similarity-based methods require access to model parameters or produce scores without thresholds.<n>We introduce Random Selection Probing (RSP), a hypothesis-testing framework that formulates model correlation detection as a statistical test.<n>RSP produces rigorous p-values that quantify evidence of correlation.
  arXiv  Detail & Related papers  (2025-09-29T01:40:26Z)
• Model-based causal feature selection for general response types [8.228587135343071]
  Invariant causal prediction (ICP) is a method for causal feature selection which requires data from heterogeneous settings. We develop transformation-model (TRAM) based ICP, allowing for continuous, categorical, count-type, and uninformatively censored responses. We provide an open-source R package 'tramicp' and evaluate our approach on simulated data and in a case study investigating causal features of survival in critically ill patients.
  arXiv  Detail & Related papers  (2023-09-22T12:42:48Z)
• TREEMENT: Interpretable Patient-Trial Matching via Personalized Dynamic Tree-Based Memory Network [54.332862955411656]
  Clinical trials are critical for drug development but often suffer from expensive and inefficient patient recruitment. In recent years, machine learning models have been proposed for speeding up patient recruitment via automatically matching patients with clinical trials. We introduce a dynamic tree-based memory network model named TREEMENT to provide accurate and interpretable patient trial matching.
  arXiv  Detail & Related papers  (2023-07-19T12:35:09Z)
• A Conditional Flow Variational Autoencoder for Controllable Synthesis of Virtual Populations of Anatomy [76.20367415712867]
  We propose a conditional variational autoencoder (cVAE) with normalising flows to boost the flexibility and complexity of the approximate posterior learnt. We demonstrate the performance of our conditional flow VAE using a data set of cardiac left ventricles acquired from 2360 patients.
  arXiv  Detail & Related papers  (2023-06-26T13:23:52Z)
• Stateful Offline Contextual Policy Evaluation and Learning [88.9134799076718]
  We study off-policy evaluation and learning from sequential data. We formalize the relevant causal structure of problems such as dynamic personalized pricing. We show improved out-of-sample policy performance in this class of relevant problems.
  arXiv  Detail & Related papers  (2021-10-19T16:15:56Z)
• A comprehensive comparative evaluation and analysis of Distributional Semantic Models [61.41800660636555]
  We perform a comprehensive evaluation of type distributional vectors, either produced by static DSMs or obtained by averaging the contextualized vectors generated by BERT. The results show that the alleged superiority of predict based models is more apparent than real, and surely not ubiquitous. We borrow from cognitive neuroscience the methodology of Representational Similarity Analysis (RSA) to inspect the semantic spaces generated by distributional models.
  arXiv  Detail & Related papers  (2021-05-20T15:18:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.