ChildEval: When large language models meet children's personalities

• URL: http://arxiv.org/abs/2605.27805v1
• Date: Wed, 27 May 2026 00:53:42 GMT
• Title: ChildEval: When large language models meet children's personalities
• Abstract summary: We introduce ChildEval, a benchmark for evaluating LLMs' ability to infer and follow child-centered preferences in long-context conversations.<n>ChildEval contains 29K synthesized persona profiles of children aged 3-6, providing relatively static background information.<n>We propose fine-grained, child-centric evaluation protocols to systematically assess open-source LLMs.
• Score: 27.061986552073012
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: While LLMs enable personalized chatbots, their effectiveness in child-centered personalization remains unclear, as systematic evaluation of child-specific preferences is still lacking. To address this gap, we introduce ChildEval, a benchmark for evaluating LLMs' ability to infer and follow child-centered preferences in long-context conversations. ChildEval contains 29K synthesized persona profiles of children aged 3-6, providing relatively static background information. Each persona is associated with a child preference-which may align with, conflict with, or be independent of the persona-expressed either explicitly in a single sentence or implicitly through 6-10 turn dialogues. Explicit and implicit preferences are designed to reflect the same underlying preference but differ in expression, capturing dynamic aspects of preference expression rather than changes in the static persona. The benchmark spans five top-level and fourteen sub-level categories covering children's daily lives and development. We further propose fine-grained, child-centric evaluation protocols to systematically assess open-source LLMs. Experimental results demonstrate how different personalized representations affect LLM responses and suggest that finetuning on ChildEval can enhance child-centered performance. Our code and dataset are available at https://github.com/ziyanluo/ChildEval.

Related papers

• ReLay: Personalized LLM-Generated Plain-Language Summaries for Better Understanding, but at What Cost? [54.44351107428039]
  Large language models (LLMs) offer new opportunities for personalizing Plain Language Summaries (PLS)<n>It remains unclear whether personalization helps, which strategies are most effective, and how to balance personalization with safety.<n>We introduce ReLay, a dataset of 300 participant--PLS pairs from 50 lay participants in both static (expert-written) and interactive settings.<n>ReLay includes user characteristics, health information needs, information-seeking behavior, comprehension outcomes, interaction logs, and quality ratings.
  arXiv  Detail & Related papers  (2026-05-01T07:11:12Z)
• Baby Scale: Investigating Models Trained on Individual Children's Language Input [2.3226022042424934]
  Modern language models must be trained on many orders of magnitude more words of training data than human children receive.<n>We benchmark LMs on human-scale datasets to understand how linguistic knowledge emerges from children's natural training data.<n>We find that LMs trained on child data show acceptable scaling for grammar tasks, but lower scaling on semantic and world knowledge tasks than models trained on synthetic data.
  arXiv  Detail & Related papers  (2026-03-31T10:06:24Z)
• Towards Realistic Personalization: Evaluating Long-Horizon Preference Following in Personalized User-LLM Interactions [50.70965714314064]
  Large Language Models (LLMs) are increasingly serving as personal assistants, where users share complex and diverse preferences over extended interactions.<n>This work proposes RealPref, a benchmark for evaluating realistic preference-following in personalized user-LLM interactions.
  arXiv  Detail & Related papers  (2026-03-04T15:42:43Z)
• Evaluating LLMs on Generating Age-Appropriate Child-Like Conversations [3.660458463669403]
  Large Language Models (LLMs) predominantly trained on adult conversational data, face challenges when generating authentic, child-like dialogue for specialized applications.<n>We present a comparative study evaluating five different LLMs to generate age-appropriate Norwegian conversations for children aged 5 and 9 years.
  arXiv  Detail & Related papers  (2025-10-28T10:00:52Z)
• Beyond Survival: Evaluating LLMs in Social Deduction Games with Human-Aligned Strategies [54.08697738311866]
  Social deduction games like Werewolf combine language, reasoning, and strategy.<n>We curate a high-quality, human-verified multimodal Werewolf dataset containing over 100 hours of video, 32.4M utterance tokens, and 15 rule variants.<n>We propose a novel strategy-alignment evaluation that leverages the winning faction's strategies as ground truth in two stages.
  arXiv  Detail & Related papers  (2025-10-13T13:33:30Z)
• A Personalized Conversational Benchmark: Towards Simulating Personalized Conversations [112.81207927088117]
  PersonaConvBench is a benchmark for evaluating personalized reasoning and generation in multi-turn conversations with large language models (LLMs)<n>We benchmark several commercial and open-source LLMs under a unified prompting setup and observe that incorporating personalized history yields substantial performance improvements.
  arXiv  Detail & Related papers  (2025-05-20T09:13:22Z)
• Benchmarking LLMs for Mimicking Child-Caregiver Language in Interaction [6.152274140650429]
  LLMs can generate human-like dialogues, yet their ability to simulate early child-adult interactions remains largely unexplored.<n>We found that state-of-the-art LLMs can approximate child- caregiver dialogues at the word and utterance level, but they struggle to reproduce the child and caregiver's discursive patterns, exaggerate alignment, and fail to reach the level of diversity shown by humans.
  arXiv  Detail & Related papers  (2024-12-12T14:43:03Z)
• Aligning LLMs with Individual Preferences via Interaction [51.72200436159636]
  We train large language models (LLMs) that can ''interact to align''<n>We develop a multi-turn preference dataset containing 3K+ multi-turn conversations in tree structures.<n>For evaluation, we establish the ALOE benchmark, consisting of 100 carefully selected examples and well-designed metrics to measure the customized alignment performance during conversations.
  arXiv  Detail & Related papers  (2024-10-04T17:48:29Z)
• Age-Dependent Analysis and Stochastic Generation of Child-Directed Speech [10.369750912567714]
  We present an approach to model age-dependent linguistic properties of child-directed speech (CDS) using a language model (LM) trained on CDS transcripts and ages of the recipient children. We compare characteristics of the generated CDS against the real speech addressed at children of different ages, showing that the LM manages to capture age-dependent changes in CDS.
  arXiv  Detail & Related papers  (2024-05-13T12:35:10Z)
• PsyCoT: Psychological Questionnaire as Powerful Chain-of-Thought for Personality Detection [50.66968526809069]
  We propose a novel personality detection method, called PsyCoT, which mimics the way individuals complete psychological questionnaires in a multi-turn dialogue manner. Our experiments demonstrate that PsyCoT significantly improves the performance and robustness of GPT-3.5 in personality detection.
  arXiv  Detail & Related papers  (2023-10-31T08:23:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.