Related papers: Agentic Society: Merging skeleton from real world and texture from Large Language Model

Agentic Society: Merging skeleton from real world and texture from Large Language Model

URL: http://arxiv.org/abs/2409.10550v1
Date: Mon, 2 Sep 2024 08:28:19 GMT
Title: Agentic Society: Merging skeleton from real world and texture from Large Language Model
Authors: Yuqi Bai, Kun Sun, Huishi Yin,
Abstract summary: This paper explores a novel framework that leverages census data and large language models to generate virtual populations. We show that our method produces personas with variability essential for simulating diverse human behaviors in social science experiments. But the evaluation result shows that only weak sign of statistical truthfulness can be produced due to limited capability of current LLMs.
Score: 4.740886789811429
License: http://creativecommons.org/licenses/by/4.0/
Abstract: Recent advancements in large language models (LLMs) and agent technologies offer promising solutions to the simulation of social science experiments, but the availability of data of real-world population required by many of them still poses as a major challenge. This paper explores a novel framework that leverages census data and LLMs to generate virtual populations, significantly reducing resource requirements and bypassing privacy compliance issues associated with real-world data, while keeping a statistical truthfulness. Drawing on real-world census data, our approach first generates a persona that reflects demographic characteristics of the population. We then employ LLMs to enrich these personas with intricate details, using techniques akin to those in image generative models but applied to textual data. Additionally, we propose a framework for the evaluation of the feasibility of our method with respect to capability of LLMs based on personality trait tests, specifically the Big Five model, which also enhances the depth and realism of the generated personas. Through preliminary experiments and analysis, we demonstrate that our method produces personas with variability essential for simulating diverse human behaviors in social science experiments. But the evaluation result shows that only weak sign of statistical truthfulness can be produced due to limited capability of current LLMs. Insights from our study also highlight the tension within LLMs between aligning with human values and reflecting real-world complexities. Thorough and rigorous test call for further research. Our codes are released at https://github.com/baiyuqi/agentic-society.git

Related papers

GenSim: A General Social Simulation Platform with Large Language Model based Agents [111.00666003559324]
We propose a novel large language model (LLMs)-based simulation platform called textitGenSim. Our platform supports one hundred thousand agents to better simulate large-scale populations in real-world contexts. To our knowledge, GenSim represents an initial step toward a general, large-scale, and correctable social simulation platform.
arXiv Detail & Related papers (2024-10-06T05:02:23Z)
A Synthetic Dataset for Personal Attribute Inference [2.9373912230684565]
We focus on the emerging privacy threat LLMs pose -- the ability to accurately infer personal information from online texts. We generate SynthPAI, a diverse synthetic dataset of over 7800 comments manually labeled for personal attributes. We validate our dataset with a human study showing that humans barely outperform random guessing on the task of distinguishing our synthetic comments from real ones.
arXiv Detail & Related papers (2024-06-11T12:50:53Z)
From Persona to Personalization: A Survey on Role-Playing Language Agents [52.783043059715546]
Recent advancements in large language models (LLMs) have boosted the rise of Role-Playing Language Agents (RPLAs) RPLAs achieve a remarkable sense of human likeness and vivid role-playing performance. They have catalyzed numerous AI applications, such as emotional companions, interactive video games, personalized assistants and copilots.
arXiv Detail & Related papers (2024-04-28T15:56:41Z)
Is this the real life? Is this just fantasy? The Misleading Success of Simulating Social Interactions With LLMs [24.613282867543244]
Large language models (LLM) have enabled richer social simulations, allowing for the study of various social phenomena. Recent work has used a more omniscient perspective on these simulations, which is fundamentally at odds with the non-omniscient, information asymmetric interactions that involve humans and AI agents in the real world.
arXiv Detail & Related papers (2024-03-08T03:49:17Z)
BASES: Large-scale Web Search User Simulation with Large Language Model based Agents [108.97507653131917]
BASES is a novel user simulation framework with large language models (LLMs) Our simulation framework can generate unique user profiles at scale, which subsequently leads to diverse search behaviors. WARRIORS is a new large-scale dataset encompassing web search user behaviors, including both Chinese and English versions.
arXiv Detail & Related papers (2024-02-27T13:44:09Z)
LLM-driven Imitation of Subrational Behavior : Illusion or Reality? [3.2365468114603937]
Existing work highlights the ability of Large Language Models to address complex reasoning tasks and mimic human communication. We propose to investigate the use of LLMs to generate synthetic human demonstrations, which are then used to learn subrational agent policies. We experimentally evaluate the ability of our framework to model sub-rationality through four simple scenarios.
arXiv Detail & Related papers (2024-02-13T19:46:39Z)
Under the Surface: Tracking the Artifactuality of LLM-Generated Data [21.002983022237604]
This work delves into the expanding role of large language models (LLMs) in generating artificial data. To the best of our knowledge, this is the first study to aggregate various types of LLM-generated text data. Despite artificial data's capability to match human performance, this paper reveals significant hidden disparities.
arXiv Detail & Related papers (2024-01-26T07:53:27Z)
Do LLMs exhibit human-like response biases? A case study in survey design [66.1850490474361]
We investigate the extent to which large language models (LLMs) reflect human response biases, if at all. We design a dataset and framework to evaluate whether LLMs exhibit human-like response biases in survey questionnaires. Our comprehensive evaluation of nine models shows that popular open and commercial LLMs generally fail to reflect human-like behavior.
arXiv Detail & Related papers (2023-11-07T15:40:43Z)
Bias and Fairness in Large Language Models: A Survey [73.87651986156006]
We present a comprehensive survey of bias evaluation and mitigation techniques for large language models (LLMs) We first consolidate, formalize, and expand notions of social bias and fairness in natural language processing. We then unify the literature by proposing three intuitive, two for bias evaluation, and one for mitigation.
arXiv Detail & Related papers (2023-09-02T00:32:55Z)
Aligning Large Language Models with Human: A Survey [53.6014921995006]
Large Language Models (LLMs) trained on extensive textual corpora have emerged as leading solutions for a broad array of Natural Language Processing (NLP) tasks. Despite their notable performance, these models are prone to certain limitations such as misunderstanding human instructions, generating potentially biased content, or factually incorrect information. This survey presents a comprehensive overview of these alignment technologies, including the following aspects.
arXiv Detail & Related papers (2023-07-24T17:44:58Z)
Large Language Models as Zero-Shot Human Models for Human-Robot Interaction [12.455647753787442]
Large-language models (LLMs) can act as zero-shot human models for human-robot interaction. LLMs achieve performance comparable to purpose-built models. We present one case study on a simulated trust-based table-clearing task.
arXiv Detail & Related papers (2023-03-06T23:16:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.