CAREBench: A Child-Safety Risk Benchmark for Language Models

• URL: http://arxiv.org/abs/2606.29685v1
• Date: Mon, 29 Jun 2026 01:17:41 GMT
• Title: CAREBench: A Child-Safety Risk Benchmark for Language Models
• Abstract summary: CAREBench is a benchmark to assess upstream child-safety risks in language models.<n>It includes grooming and relationship engineering, deception and impersonation, surveillance and privacy, sextortion and sexual abuse.<n>It evaluates whether models recognize, refuse, de-escalate, or redirect risky interactions before harm becomes overt.
• Score: 8.997978966432546
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: How can we evaluate whether frontier AI systems recognize child-safety risks before they escalate into explicit harm? Existing child safety evaluations focus on child sexual abuse material, yet many child-safety failures begin earlier: in model assistance that helps adults manipulate, impersonate, profile, or isolate minors, and in model responses that deepen children's emotional dependence on AI systems rather than redirecting them toward human support. We introduce CAREBench (Child AI Risk Evaluation), a benchmark to assess such upstream child-safety risks in language models. CAREBench contains 500 prompts spanning twelve risk categories, including grooming and relationship engineering, deception and impersonation, surveillance and privacy, sextortion and sexual abuse, AI anthropomorphization, emotional dependency, and mental illness sensitivity. Developed with response annotations from parents and clinicians, the benchmark excludes explicit abuse material and imagery; instead, it evaluates whether models recognize, refuse, de-escalate, or redirect risky interactions before harm becomes overt. Evaluating seven frontier models on our benchmark, we find failure rates ranging from 2% to 58%, with failure patterns that vary across risk categories. CAREBench provides a responsibly scoped evaluation for LLM developers to identify and close gaps in child safety policies.

Related papers

• The Age of Curiosity Meets the Age of AI: Benchmarking Child Safety in Large Language Models [35.56441154476198]
  We introduce KIDBench, a benchmark for evaluating child-facing Large Language Models safety for ages 7-11.<n> KIDBench contains realistic child queries across ten categories, with single-turn prompts and multi-turn child-actor simulations.<n> KIDGuardLlama is a child-oriented response model, showing how KIDBench supports safer child-facing AI.
  arXiv  Detail & Related papers  (2026-05-25T07:14:58Z)
• Emergent Strategic Reasoning Risks in AI: A Taxonomy-Driven Evaluation Framework [63.74295981594549]
  Large language models (LLMs) engage in behaviors that serve their own objectives.<n>These include, but are not limited to, deception (intentionally misleading users or evaluators), evaluation gaming (strategically manipulating performance during safety testing), and reward hacking.<n>We introduce ESRRSim, a taxonomy-driven agentic framework for automated behavioral risk evaluation.
  arXiv  Detail & Related papers  (2026-04-23T23:44:01Z)
• SafetyALFRED: Evaluating Safety-Conscious Planning of Multimodal Large Language Models [46.968844120076916]
  We introduce SafetyALFRED, built upon the embodied agent benchmark ALFRED, augmented with six categories of real-world kitchen hazards.<n>We evaluate eleven state-of-the-art models on not only hazard recognition, but also active risk mitigation through embodied planning.
  arXiv  Detail & Related papers  (2026-04-21T16:27:20Z)
• CASTLE: A Comprehensive Benchmark for Evaluating Student-Tailored Personalized Safety in Large Language Models [55.0103764229311]
  We propose the concept of Student-Tailored Personalized Safety and construct CASTLE based on educational theories.<n>This benchmark covers 15 educational safety risks and 14 student attributes, comprising 92,908 bilingual scenarios.
  arXiv  Detail & Related papers  (2026-02-05T13:13:19Z)
• Toward Quantitative Modeling of Cybersecurity Risks Due to AI Misuse [50.87630846876635]
  We develop nine detailed cyber risk models.<n>Each model decomposes attacks into steps using the MITRE ATT&CK framework.<n>Individual estimates are aggregated through Monte Carlo simulation.
  arXiv  Detail & Related papers  (2025-12-09T17:54:17Z)
• Evaluating LLM Safety Across Child Development Stages: A Simulated Agent Approach [9.544657426086284]
  We present ChildSafe, a benchmark that evaluates Large Language Models (LLMs) safety through simulated child agents.<n>ChildSafe assesses responses across nine safety dimensions using age-weighted scoring in both sensitive and neutral contexts.
  arXiv  Detail & Related papers  (2025-10-07T01:01:04Z)
• Oyster-I: Beyond Refusal -- Constructive Safety Alignment for Responsible Language Models [93.5740266114488]
  Constructive Safety Alignment (CSA) protects against malicious misuse while actively guiding vulnerable users toward safe and helpful results.<n>Oy1 achieves state-of-the-art safety among open models while retaining high general capabilities.<n>We release Oy1, code, and the benchmark to support responsible, user-centered AI.
  arXiv  Detail & Related papers  (2025-09-02T03:04:27Z)
• SproutBench: A Benchmark for Safe and Ethical Large Language Models for Youth [14.569766143989531]
  The rapid proliferation of large language models (LLMs) in applications targeting children and adolescents necessitates a fundamental reassessment of prevailing AI safety frameworks.<n>This paper highlights key deficiencies in existing LLM safety benchmarks, including their inadequate coverage of age-specific cognitive, emotional, and social risks.<n>We introduce SproutBench, an innovative evaluation suite comprising 1,283 developmentally grounded adversarial prompts designed to probe risks such as emotional dependency, privacy violations, and imitation of hazardous behaviors.
  arXiv  Detail & Related papers  (2025-08-14T18:21:39Z)
• MinorBench: A hand-built benchmark for content-based risks for children [0.0]
  Large Language Models (LLMs) are rapidly entering children's lives through parent-driven adoption, schools, and peer networks.<n>Current AI ethics and safety research do not adequately address content-related risks specific to minors.<n>We propose a new taxonomy of content-based risks for minors and introduce MinorBench, an open-source benchmark designed to evaluate LLMs on their ability to refuse unsafe or inappropriate queries from children.
  arXiv  Detail & Related papers  (2025-03-13T10:34:43Z)
• AILuminate: Introducing v1.0 of the AI Risk and Reliability Benchmark from MLCommons [62.374792825813394]
  This paper introduces AILuminate v1.0, the first comprehensive industry-standard benchmark for assessing AI-product risk and reliability.<n>The benchmark evaluates an AI system's resistance to prompts designed to elicit dangerous, illegal, or undesirable behavior in 12 hazard categories.
  arXiv  Detail & Related papers  (2025-02-19T05:58:52Z)
• Toward Improving Predictive Risk Modelling for New Zealand's Child Welfare System Using Clustering Methods [0.0]
  We aim to discover the extent of clustering degree required as an early step in the development of predictive risk models for child maltreatment. Our results suggest that separate models might need to be developed for children of certain age to gain additional control over the error rates and to improve model accuracy.
  arXiv  Detail & Related papers  (2023-08-08T05:46:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.