AIriskEval-edu: New Dataset for Risk Assessment in AI-mediated K-12 Educational Explanations

• URL: http://arxiv.org/abs/2607.01934v1
• Date: Thu, 02 Jul 2026 09:28:21 GMT
• Title: AIriskEval-edu: New Dataset for Risk Assessment in AI-mediated K-12 Educational Explanations
• Abstract summary: dataset comprises 1,639 explanations from 170 curated ScienceQA questions, covering science, language arts, and social sciences.<n>Key contribution is the addition of 785 explanations with structured explainability annotations, including risk localization and risk description.<n>Experiments evaluate whether supervised fine-tuning on AIriskEval-edu-db2 enables a locally deployable model to approach or outperform stronger frontier models.
• Score: 12.641424626110899
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This work introduces AIriskEval-edu-db2, a new dataset designed to train and evaluate auditors based on LLMs for an explainable pedagogical risk assessment in instructional content for grades K-12. The dataset comprises 1,639 explanations from 170 curated ScienceQA questions, covering science, language arts, and social sciences. For each question, the dataset includes an explanation written by a human teacher alongside 11 explanations generated by LLM-simulated teacher profiles associated with distinct pedagogical risks. We propose a comprehensive risk rubric aligned with established educational standards that covers five complementary dimensions: factual precision, depth and completeness, focus and relevance, student-level appropriateness, and ideological bias. A key contribution is the addition of 785 explanations with structured explainability annotations, including risk localization and risk description. The annotations are produced through a semi-automatic process with expert teacher validation. Finally, we present validation experiments comparing state-of-the-art proprietary models with a lightweight local Llama 3.1 8B model in both the pedagogical risk detection and the explainability assessment. These experiments evaluate whether supervised fine-tuning on AIriskEval-edu-db2 enables a locally deployable model to approach or outperform stronger frontier models while preserving privacy in educational auditing and assessment tasks.

Related papers

• AIriskEval-edu Demo: Auditing of Pedagogical Risks in Educational Explanations [12.641424626110899]
  We present AIriskEval-edu, a platform that audits the pedagogical quality of instructional explanations.<n>The platform evaluates an explanation against a rubric covering five dimensions of pedagogical risk.<n>For each dimension, it returns a binary decision and a confidence score.
  arXiv  Detail & Related papers  (2026-07-28T12:16:28Z)
• Risk-Consistent Multiclass Learning from Random Label-Subset Membership Queries [23.692455958661668]
  This paper proposes a multiclass learn ing framework under random label-subset queries.<n>We derive an unbiased estimator of the target risk under the empirical risk minimization (ERM) framework.<n>To address negative empirical risk and the associated overfitting problem, we introduce corrected risk estimators based on non-negative and absolute-value corrections.
  arXiv  Detail & Related papers  (2026-05-08T08:05:02Z)
• Emergent Strategic Reasoning Risks in AI: A Taxonomy-Driven Evaluation Framework [63.74295981594549]
  Large language models (LLMs) engage in behaviors that serve their own objectives.<n>These include, but are not limited to, deception (intentionally misleading users or evaluators), evaluation gaming (strategically manipulating performance during safety testing), and reward hacking.<n>We introduce ESRRSim, a taxonomy-driven agentic framework for automated behavioral risk evaluation.
  arXiv  Detail & Related papers  (2026-04-23T23:44:01Z)
• EduEVAL-DB: A Role-Based Dataset for Pedagogical Risk Evaluation in Educational Explanations [13.185893390841693]
  dataset comprises 854 explanations corresponding to 139 questions from a curated subset of the ScienceQA benchmark.<n>We propose a pedagogical risk aligned with established educational standards, operationalizing five complementary risk dimensions.<n>We benchmark a state-of-the-art education-oriented model (Gemini 2.5 Pro) against a lightweight local Llama 3.1 8B model.
  arXiv  Detail & Related papers  (2026-02-17T12:11:49Z)
• There Are No Silly Questions: Evaluation of Offline LLM Capabilities from a Turkish Perspective [0.0]
  This study aims to evaluate the robustness and pedagogical safety of locally deployable offline LLMs within the context of Turkish heritage language education.<n>Experiments conducted on 14 different models ranging from 270M to 32B parameters reveal that anomaly resistance is not solely dependent on model scale.<n>The findings indicate that reasoning-oriented models in the 8B--14B parameter range represent the most balanced segment in terms of cost-safety trade-off for language learners.
  arXiv  Detail & Related papers  (2026-02-15T09:49:45Z)
• CASTLE: A Comprehensive Benchmark for Evaluating Student-Tailored Personalized Safety in Large Language Models [55.0103764229311]
  We propose the concept of Student-Tailored Personalized Safety and construct CASTLE based on educational theories.<n>This benchmark covers 15 educational safety risks and 14 student attributes, comprising 92,908 bilingual scenarios.
  arXiv  Detail & Related papers  (2026-02-05T13:13:19Z)
• RADAR: A Risk-Aware Dynamic Multi-Agent Framework for LLM Safety Evaluation via Role-Specialized Collaboration [81.38705556267917]
  Existing safety evaluation methods for large language models (LLMs) suffer from inherent limitations.<n>We introduce a theoretical framework that reconstructs the underlying risk concept space.<n>We propose RADAR, a multi-agent collaborative evaluation framework.
  arXiv  Detail & Related papers  (2025-09-28T09:35:32Z)
• STRICTA: Structured Reasoning in Critical Text Assessment for Peer Review and Beyond [68.47402386668846]
  We introduce Structured Reasoning In Critical Text Assessment (STRICTA) to model text assessment as an explicit, step-wise reasoning process.<n>STRICTA breaks down the assessment into a graph of interconnected reasoning steps drawing on causality theory.<n>We apply STRICTA to a dataset of over 4000 reasoning steps from roughly 40 biomedical experts on more than 20 papers.
  arXiv  Detail & Related papers  (2024-09-09T06:55:37Z)
• Unveiling the Misuse Potential of Base Large Language Models via In-Context Learning [61.2224355547598]
  Open-sourcing of large language models (LLMs) accelerates application development, innovation, and scientific progress. Our investigation exposes a critical oversight in this belief. By deploying carefully designed demonstrations, our research demonstrates that base LLMs could effectively interpret and execute malicious instructions.
  arXiv  Detail & Related papers  (2024-04-16T13:22:54Z)
• Informing clinical assessment by contextualizing post-hoc explanations of risk prediction models in type-2 diabetes [50.8044927215346]
  We consider a comorbidity risk prediction scenario and focus on contexts regarding the patients clinical state. We employ several state-of-the-art LLMs to present contexts around risk prediction model inferences and evaluate their acceptability. Our paper is one of the first end-to-end analyses identifying the feasibility and benefits of contextual explanations in a real-world clinical use case.
  arXiv  Detail & Related papers  (2023-02-11T18:07:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.