AIriskEval-edu Demo: Auditing of Pedagogical Risks in Educational Explanations

• URL: http://arxiv.org/abs/2607.25634v1
• Date: Tue, 28 Jul 2026 12:16:28 GMT
• Title: AIriskEval-edu Demo: Auditing of Pedagogical Risks in Educational Explanations
• Abstract summary: We present AIriskEval-edu, a platform that audits the pedagogical quality of instructional explanations.<n>The platform evaluates an explanation against a rubric covering five dimensions of pedagogical risk.<n>For each dimension, it returns a binary decision and a confidence score.
• Score: 12.641424626110899
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present AIriskEval-edu Demo, a platform that audits the pedagogical quality of instructional explanations and provides explainable audit results. The platform evaluates an explanation against a rubric covering five dimensions of pedagogical risk: factual accuracy, depth and completeness, focus and relevance, student-level appropriateness, and ideological bias. For each dimension, it returns a binary decision and a confidence score. Detected risks also include a natural-language rationale and, except for Depth and Completeness, a localized evidence span. The platform integrates GPT-5.5 through an external API and a self-hosted Llama 3.1 8B evaluator that runs on consumer-grade GPUs. The local evaluator is fine-tuned on AIriskEval-edu, a dataset of K-12 instructional explanations with risk and explainability annotations. The platform operates in two modes: in AI mode, both evaluators assess stored explanations generated under six simulated teacher profiles, each representing a distinct pedagogical behavior and potential risk; in human mode, the local evaluator audits user-written explanations in real time. The local evaluator outperforms GPT-5.5 on most reported metrics, offering educational institutions a practical way to keep audited content within their own infrastructure.

Related papers

• Who Should Grade My Work? Student Perspectives on Transparent AI-Assisted Writing Assessment in Higher Education [0.0]
  The integration of GenAI tools into higher education assessment raises important questions about how students understand, interpret, and respond to AI-mediated evaluation.<n>This study reports findings from a qualitative pedagogical inquiry conducted in an undergraduate technical communication course for computing students at a Saudi public university.<n>Students were explicitly informed that ChatGPT had generated the score and feedback and were invited to reflect on the evaluation in writing.
  arXiv  Detail & Related papers  (2026-09-04T16:49:33Z)
• AIriskEval-edu: New Dataset for Risk Assessment in AI-mediated K-12 Educational Explanations [12.641424626110899]
  dataset comprises 1,639 explanations from 170 curated ScienceQA questions, covering science, language arts, and social sciences.<n>Key contribution is the addition of 785 explanations with structured explainability annotations, including risk localization and risk description.<n>Experiments evaluate whether supervised fine-tuning on AIriskEval-edu-db2 enables a locally deployable model to approach or outperform stronger frontier models.
  arXiv  Detail & Related papers  (2026-07-02T09:28:21Z)
• EduEVAL-DB: A Role-Based Dataset for Pedagogical Risk Evaluation in Educational Explanations [13.185893390841693]
  dataset comprises 854 explanations corresponding to 139 questions from a curated subset of the ScienceQA benchmark.<n>We propose a pedagogical risk aligned with established educational standards, operationalizing five complementary risk dimensions.<n>We benchmark a state-of-the-art education-oriented model (Gemini 2.5 Pro) against a lightweight local Llama 3.1 8B model.
  arXiv  Detail & Related papers  (2026-02-17T12:11:49Z)
• CASTLE: A Comprehensive Benchmark for Evaluating Student-Tailored Personalized Safety in Large Language Models [55.0103764229311]
  We propose the concept of Student-Tailored Personalized Safety and construct CASTLE based on educational theories.<n>This benchmark covers 15 educational safety risks and 14 student attributes, comprising 92,908 bilingual scenarios.
  arXiv  Detail & Related papers  (2026-02-05T13:13:19Z)
• Auditing Language Model Unlearning via Information Decomposition [68.48660428111593]
  We introduce an interpretable, information-theoretic framework for auditing unlearning using Partial Information Decomposition (PID)<n>By comparing model representations before and after unlearning, we decompose the mutual information with the forgotten data into distinct components, formalizing the notions of unlearned and residual knowledge.<n>Our work introduces a principled, representation-level audit for unlearning, offering theoretical insight and actionable tools for safer deployment of language models.
  arXiv  Detail & Related papers  (2026-01-21T15:51:19Z)
• ELI-Why: Evaluating the Pedagogical Utility of Language Model Explanations [38.73656006445607]
  We introduce ELI-Why, a benchmark of 13.4K "Why" questions to evaluate the pedagogical capabilities of language models.<n>In our first study, human raters assume the role of an "educator" to assess model explanations' fit to different educational grades.<n>We find that GPT-4-generated explanations match their intended educational background only 50% of the time, compared to 79% for lay human-curated explanations.
  arXiv  Detail & Related papers  (2025-06-17T05:36:39Z)
• MathTutorBench: A Benchmark for Measuring Open-ended Pedagogical Capabilities of LLM Tutors [82.91830877219822]
  We present MathTutorBench, an open-source benchmark for holistic tutoring model evaluation.<n>MathTutorBench contains datasets and metrics that broadly cover tutor abilities as defined by learning sciences research in dialog-based teaching.<n>We evaluate a wide set of closed- and open-weight models and find that subject expertise, indicated by solving ability, does not immediately translate to good teaching.
  arXiv  Detail & Related papers  (2025-02-26T08:43:47Z)
• Do Tutors Learn from Equity Training and Can Generative AI Assess It? [2.116573423199236]
  We evaluate tutor performance within an online lesson on enhancing tutors' skills when responding to students in potentially inequitable situations.<n>We find marginally significant learning gains with increases in tutors' self-reported confidence in their knowledge.<n>This work makes available a dataset of lesson log data, tutor responses, rubrics for human annotation, and generative AI prompts.
  arXiv  Detail & Related papers  (2024-12-15T17:36:40Z)
• On the Loss of Context-awareness in General Instruction Fine-tuning [101.03941308894191]
  We investigate the loss of context awareness after supervised fine-tuning.<n>We find that the performance decline is associated with a bias toward different roles learned during conversational instruction fine-tuning.<n>We propose a metric to identify context-dependent examples from general instruction fine-tuning datasets.
  arXiv  Detail & Related papers  (2024-11-05T00:16:01Z)
• Evaluating the Utility of Model Explanations for Model Development [54.23538543168767]
  We evaluate whether explanations can improve human decision-making in practical scenarios of machine learning model development. To our surprise, we did not find evidence of significant improvement on tasks when users were provided with any of the saliency maps. These findings suggest caution regarding the usefulness and potential for misunderstanding in saliency-based explanations.
  arXiv  Detail & Related papers  (2023-12-10T23:13:23Z)
• Bring Your Own Data! Self-Supervised Evaluation for Large Language Models [52.15056231665816]
  We propose a framework for self-supervised evaluation of Large Language Models (LLMs) We demonstrate self-supervised evaluation strategies for measuring closed-book knowledge, toxicity, and long-range context dependence. We find strong correlations between self-supervised and human-supervised evaluations.
  arXiv  Detail & Related papers  (2023-06-23T17:59:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.