EduEVAL-DB: A Role-Based Dataset for Pedagogical Risk Evaluation in Educational Explanations

• URL: http://arxiv.org/abs/2602.15531v2
• Date: Thu, 19 Feb 2026 12:33:08 GMT
• Title: EduEVAL-DB: A Role-Based Dataset for Pedagogical Risk Evaluation in Educational Explanations
• Abstract summary: dataset comprises 854 explanations corresponding to 139 questions from a curated subset of the ScienceQA benchmark.<n>We propose a pedagogical risk aligned with established educational standards, operationalizing five complementary risk dimensions.<n>We benchmark a state-of-the-art education-oriented model (Gemini 2.5 Pro) against a lightweight local Llama 3.1 8B model.
• Score: 13.185893390841693
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This work introduces EduEVAL-DB, a dataset based on teacher roles designed to support the evaluation and training of automatic pedagogical evaluators and AI tutors for instructional explanations. The dataset comprises 854 explanations corresponding to 139 questions from a curated subset of the ScienceQA benchmark, spanning science, language, and social science across K-12 grade levels. For each question, one human-teacher explanation is provided and six are generated by LLM-simulated teacher roles. These roles are inspired by instructional styles and shortcomings observed in real educational practice and are instantiated via prompt engineering. We further propose a pedagogical risk rubric aligned with established educational standards, operationalizing five complementary risk dimensions: factual correctness, explanatory depth and completeness, focus and relevance, student-level appropriateness, and ideological bias. All explanations are annotated with binary risk labels through a semi-automatic process with expert teacher review. Finally, we present preliminary validation experiments to assess the suitability of EduEVAL-DB for evaluation. We benchmark a state-of-the-art education-oriented model (Gemini 2.5 Pro) against a lightweight local Llama 3.1 8B model and examine whether supervised fine-tuning on EduEVAL-DB supports pedagogical risk detection using models deployable on consumer hardware.

Related papers

• AIriskEval-edu Demo: Auditing of Pedagogical Risks in Educational Explanations [12.641424626110899]
  We present AIriskEval-edu, a platform that audits the pedagogical quality of instructional explanations.<n>The platform evaluates an explanation against a rubric covering five dimensions of pedagogical risk.<n>For each dimension, it returns a binary decision and a confidence score.
  arXiv  Detail & Related papers  (2026-07-28T12:16:28Z)
• AIriskEval-edu: New Dataset for Risk Assessment in AI-mediated K-12 Educational Explanations [12.641424626110899]
  dataset comprises 1,639 explanations from 170 curated ScienceQA questions, covering science, language arts, and social sciences.<n>Key contribution is the addition of 785 explanations with structured explainability annotations, including risk localization and risk description.<n>Experiments evaluate whether supervised fine-tuning on AIriskEval-edu-db2 enables a locally deployable model to approach or outperform stronger frontier models.
  arXiv  Detail & Related papers  (2026-07-02T09:28:21Z)
• Measuring Whether LLM Tutors Teach or Solve: A Diagnostic for Educational Impact [4.6840536642016195]
  We study whether public tutoring benchmarks distinguish learning-supportive behavior from mere answer production.<n>We propose a lightweight diagnostic rubric based on the gap between solving-oriented and pedagogy-oriented benchmark performance.
  arXiv  Detail & Related papers  (2026-06-15T04:32:18Z)
• Are Agents Ready to Teach? A Multi-Stage Benchmark for Real-World Teaching Workflows [48.61619205237941]
  EduAgentBench is a source-grounded benchmark for evaluating tutor agents across the full scope of teaching work.<n>It contains 150 quality-controlled tasks across three capability surfaces: professional pedagogical judgment, situated multi-turn tutoring, and Canvas-style teaching workflow completion.<n>To our knowledge, EduAgentBench is the first theory-grounded and realistic benchmark for evaluating the holistic teaching capability of tutor agents.
  arXiv  Detail & Related papers  (2026-05-14T03:34:25Z)
• Pedagogical Alignment for Vision-Language-Action Models: A Comprehensive Framework for Data, Architecture, and Evaluation in Education [3.827767386780446]
  We present textitPedagogical VLA Framework, a framework that applies pedagogical alignment to lightweight VLA models.<n>We evaluate Pedagogical VLA Framework across five science demonstrations spanning physics, chemistry, biology, and earth science.
  arXiv  Detail & Related papers  (2026-01-20T11:43:15Z)
• Foundational Automatic Evaluators: Scaling Multi-Task Generative Evaluator Training for Reasoning-Centric Domains [97.5573252172065]
  We train a family of Automatic Reasoning Evaluators (FARE) with a simple iterative rejection-sampling supervised finetuning approach.<n>FARE-8B challenges larger specialized RL-trained evaluators and FARE-20B sets the new standard for open-source evaluators.<n>As inference-time rerankers, FARE-20B achieves near-oracle performance on MATH.
  arXiv  Detail & Related papers  (2025-10-20T17:52:06Z)
• EduDial: Constructing a Large-scale Multi-turn Teacher-Student Dialogue Corpus [59.693733170193944]
  We present EduDial, a comprehensive multi-turn teacher-student dialogue dataset.<n>EduDial covers 345 core knowledge points and consists of 34,250 dialogue sessions generated through interactions between teacher and student agents.
  arXiv  Detail & Related papers  (2025-10-14T18:18:43Z)
• Decoding Instructional Dialogue: Human-AI Collaborative Analysis of Teacher Use of AI Tool at Scale [9.092920230987684]
  The integration of large language models into educational tools has the potential to substantially impact how teachers plan instruction.<n>This paper presents a human-AI collaborative methodology for large-scale qualitative analysis of over 140,000 educator-AI messages.
  arXiv  Detail & Related papers  (2025-07-23T23:23:38Z)
• Benchmarking the Pedagogical Knowledge of Large Language Models [4.417539128489408]
  This paper introduces The Pedagogy Benchmark, a novel dataset designed to evaluate large language models on their pedagogical knowledge.<n>These benchmarks are built on a carefully curated set of questions sourced from professional development exams for teachers.<n>We report results for 97 models, with accuracies spanning a range from 28% to 89% on the pedagogical knowledge questions.
  arXiv  Detail & Related papers  (2025-06-23T14:49:01Z)
• Pedagogy-R1: Pedagogically-Aligned Reasoning Model with Balanced Educational Benchmark [6.024228339466189]
  Large reasoning models (LRMs) show strong performance in structured domains such as mathematics and programming.<n>LRMs often lack pedagogical coherence and realistic teaching behaviors.<n>We introduce Pedagogy-R1, a framework that adapts LRMs for classroom use through three innovations.
  arXiv  Detail & Related papers  (2025-05-24T02:18:35Z)
• MathTutorBench: A Benchmark for Measuring Open-ended Pedagogical Capabilities of LLM Tutors [82.91830877219822]
  We present MathTutorBench, an open-source benchmark for holistic tutoring model evaluation.<n>MathTutorBench contains datasets and metrics that broadly cover tutor abilities as defined by learning sciences research in dialog-based teaching.<n>We evaluate a wide set of closed- and open-weight models and find that subject expertise, indicated by solving ability, does not immediately translate to good teaching.
  arXiv  Detail & Related papers  (2025-02-26T08:43:47Z)
• HREF: Human Response-Guided Evaluation of Instruction Following in Language Models [61.273153125847166]
  We develop a new evaluation benchmark, Human Response-Guided Evaluation of Instruction Following (HREF)<n>In addition to providing reliable evaluation, HREF emphasizes individual task performance and is free from contamination.<n>We study the impact of key design choices in HREF, including the size of the evaluation set, the judge model, the baseline model, and the prompt template.
  arXiv  Detail & Related papers  (2024-12-20T03:26:47Z)
• Unifying AI Tutor Evaluation: An Evaluation Taxonomy for Pedagogical Ability Assessment of LLM-Powered AI Tutors [7.834688858839734]
  We investigate whether current state-of-the-art large language models (LLMs) are effective as AI tutors.<n>We propose a unified evaluation taxonomy with eight pedagogical dimensions based on key learning sciences principles.<n>We release MRBench - a new evaluation benchmark containing 192 conversations and 1,596 responses from seven state-of-the-art LLM-based and human tutors.
  arXiv  Detail & Related papers  (2024-12-12T16:24:35Z)
• The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Models [94.31327813151208]
  BiGGen Bench is a principled generation benchmark designed to thoroughly evaluate nine distinct capabilities of LMs across 77 diverse tasks.<n>A key feature of the BiGGen Bench is its use of instance-specific evaluation criteria, closely mirroring the nuanced discernment of human evaluation.
  arXiv  Detail & Related papers  (2024-06-09T12:30:30Z)
• Distantly-Supervised Named Entity Recognition with Adaptive Teacher Learning and Fine-grained Student Ensemble [56.705249154629264]
  Self-training teacher-student frameworks are proposed to improve the robustness of NER models. In this paper, we propose an adaptive teacher learning comprised of two teacher-student networks. Fine-grained student ensemble updates each fragment of the teacher model with a temporal moving average of the corresponding fragment of the student, which enhances consistent predictions on each model fragment against noise.
  arXiv  Detail & Related papers  (2022-12-13T12:14:09Z)
• Learning Action Conditions from Instructional Manuals for Instruction Understanding [48.52663250368341]
  We propose a task dubbed action condition inference, and collecting a high-quality, human annotated dataset of preconditions and postconditions of actions in instructional manuals. We propose a weakly supervised approach to automatically construct large-scale training instances from online instructional manuals, and curate a densely human-annotated and validated dataset to study how well the current NLP models can infer action-condition dependencies in instruction texts.
  arXiv  Detail & Related papers  (2022-05-25T00:19:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.