EduGuard: A Safe RAG-Based LLM Tutor for Programming Education

• URL: http://arxiv.org/abs/2607.15738v1
• Date: Fri, 17 Jul 2026 08:18:18 GMT
• Title: EduGuard: A Safe RAG-Based LLM Tutor for Programming Education
• Abstract summary: Generative AI (GenAI) is increasingly used by students for programming explanation, debug, and assignment support.<n>This paper presents EduGuard, a safe retrieval-augmented generation (RAG) tutoring framework for introductory programming.<n>EduGuard integrates query understanding, instructor-approved course retrieval, pedagogical strategy selection, rubric-aware generation, claim-level verification, and overreliance control.
• Score: 1.2999518604217852
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generative AI (GenAI) is increasingly used by students for programming explanation, debugging, and assignment support. Yet unrestricted large language model (LLM) tutors can hallucinate, contradict course policy, reveal complete solutions, and foster passive dependence. This paper presents EduGuard, a safe retrieval-augmented generation (RAG) tutoring framework for introductory programming. EduGuard integrates query understanding, instructor-approved course retrieval, pedagogical strategy selection, rubric-aware generation, claim-level verification, and overreliance control. To make evaluation provenance explicit, we construct BILearn-CS, a 600-query instructor-authored, TA-validated benchmark spanning concept questions, debugging cases, misconceptions, assignment-support requests, code-mixed Bangla-English queries, and adversarial direct-answer prompts. Moving beyond a synthetic-only benchmark, we further evaluate on a 150-query public CS50-style course-forum set and run a small controlled pilot with 10 undergraduates using a counterbalanced pre-test/post-test design. Using Meta-Llama-3.1-8B-Instruct as the primary generator, hybrid FAISS/BM25 retrieval, and DeBERTa-v3-large-MNLI as an architecturally separate verifier, EduGuard is compared against strong baselines: GPT-4o-mini Tutor, Llama Socratic Tutor, LPITutor-style RAG, RAG with rubric prompting, and RAG with same-model self-checking. On BILearn-CS, EduGuard attains the best correctness (90.1%), grounding (89.4%), and rubric alignment (90.8%), with the lowest hallucination (4.9%) and direct-answer leakage (9.8%). In the pilot, it raises immediate post-test accuracy from 68.4% to 81.2% and cuts overreliance from 38.0% to 17.0% relative to GPT-4o-mini Tutor. These results suggest safe GenAI tutoring requires not only retrieval or strong prompting, but explicit pedagogical control, evidence verification, and deployment safeguards.

Related papers

• Evaluating and Improving Pedagogical Fit in LLM-Based AI Tutors with the Pedagogical Suitability Index [0.26508608365976566]
  The Pedagogical Suitability Index (PSI) is a composite metric of six theory-informed sub-scores.<n>We evaluate four large language models (LLMs) across 240 scenario-based evaluations.<n>PSI-guided feedback substantially improved weak-performing cases.
  arXiv  Detail & Related papers  (2026-08-05T21:07:10Z)
• Knowledge Distillation for Automated AI Tutor Evaluation [0.0]
  We introduce FATE (FLC AI Tutor Evaluator), a specialized language model designed to evaluate AI tutors.<n>Aligned with the four core evaluation tracks from the BEA 2025 Shared Task, our model assesses pedagogical ability across Mistake Identification, Mistake Location, Guidance, and Actionability.<n>We demonstrate FATE's utility as an automated evaluator by benchmarking instructional responses generated by popular commercial models, including ChatGPT, Claude, Gemini, and DeepSeek.
  arXiv  Detail & Related papers  (2026-07-12T08:28:11Z)
• Compete Then Collaborate: Frontier AI Teachers Build a Verifiable Curriculum to Improve a Coding Student Beyond Imitation [0.0]
  We introduce a compete-then-collaborate framework where four frontier AI teachers are ranked head-to-head by an execution-based judge.<n>Using the same collaborative curriculum as a reinforcement learning with verifiable rewards (RLVR) environment improves the student.
  arXiv  Detail & Related papers  (2026-07-09T09:00:52Z)
• Mastermind: Strategy-grounded Learning for Repository-Scale Vulnerability Reproduction [62.42945715963878]
  Repository-level vulnerability reproduction is a demanding software engineering (SE) task.<n>Recent LLM agents can often execute these steps when the approach is correct, yet they still fail by choosing the wrong strategy.<n>This paper argues that strategy, rather than the full action trajectory, is the right learning unit for such SE agents.
  arXiv  Detail & Related papers  (2026-07-02T06:27:27Z)
• ACCORD: Action-Conditioned Contextual Grounding for Language Agents [69.73525608707764]
  ACCORD (Action-Conditioned Contextual Grounding) is an agent framework for adaptive grounding.<n>We show that current agents often fail to do so. They act from assumed rather than observed specifics, overlook information they could have gathered, and fail to incorporate evidence that has already been returned.
  arXiv  Detail & Related papers  (2026-06-15T09:05:55Z)
• GRADE: Generalizable Reasoning-Aware Dialogue Evaluation for AI Tutors [5.831342304669597]
  GRADE is a systematic study of open-source models for pedagogical ability assessment in student-tutor dialogues.<n>We evaluate 120 configurations across five language models, zero-shot inference, LoRA fine-tuning, synthetic augmentation, CoT+Reasoning, and single-task versus multitask formulations.
  arXiv  Detail & Related papers  (2026-05-27T02:26:55Z)
• Evaluating Prompt Injection Defenses for Educational LLM Tutors: Security-Usability-Latency Trade-offs [51.56484100374058]
  guardrail design entails explicit trade-offs among adversarial robustness, benign-task usability, and response latency.<n>We evaluate a domain-specific multi-layer safeguard pipeline combining deterministic pattern filters, structural validation, contextual sandboxing, and session-level behavioral checks.<n>Results expose operational trade-offs: NeMo reaches 0 percent bypass at 16.22 percent FPR and roughly 1.5s latency, while Prompt Guard yields 38.48 percent bypass with 3.60 percent FPR.
  arXiv  Detail & Related papers  (2026-03-29T18:52:01Z)
• GATES: Self-Distillation under Privileged Context with Consensus Gating [89.62339954332248]
  We study self-distillation in settings where supervision is unreliable.<n>We focus on document-grounded question answering with asymmetric context.<n>We derive supervision online from tutor consensus by sampling multiple document-grounded reasoning traces.
  arXiv  Detail & Related papers  (2026-02-24T05:56:20Z)
• TutorBench: A Benchmark To Assess Tutoring Capabilities Of Large Language Models [10.963195858672627]
  TutorBench is a dataset and evaluation benchmark designed to rigorously evaluate the core tutoring skills of large language models (LLMs)<n>Samples are drawn from three common tutoring tasks: (i) generating adaptive explanations tailored to a student's confusion, (ii) providing actionable feedback on a student's work, and (iii) promoting active learning through effective hint generation.<n>We evaluate 16 frontier LLMs on TutorBench and present a detailed analysis of their performance and behavior.
  arXiv  Detail & Related papers  (2025-10-03T01:41:09Z)
• AegisLLM: Scaling Agentic Systems for Self-Reflective Defense in LLM Security [74.22452069013289]
  AegisLLM is a cooperative multi-agent defense against adversarial attacks and information leakage.<n>We show that scaling agentic reasoning system at test-time substantially enhances robustness without compromising model utility.<n> Comprehensive evaluations across key threat scenarios, including unlearning and jailbreaking, demonstrate the effectiveness of AegisLLM.
  arXiv  Detail & Related papers  (2025-04-29T17:36:05Z)
• How to Build an Adaptive AI Tutor for Any Course Using Knowledge Graph-Enhanced Retrieval-Augmented Generation (KG-RAG) [5.305156933641317]
  Large Language Models (LLMs) in Intelligent Tutoring Systems (ITS) presents transformative opportunities for personalized education.<n>Current implementations face two critical challenges: maintaining factual accuracy and delivering coherent, context-aware instruction.<n>This paper introduces Knowledge Graph-enhanced Retrieval-Augmented Generation (RAG), a novel framework that integrates structured knowledge representation with context-aware retrieval.
  arXiv  Detail & Related papers  (2023-11-29T15:02:46Z)
• LGD: Label-guided Self-distillation for Object Detection [59.9972914042281]
  We propose the first self-distillation framework for general object detection, termed LGD (Label-Guided self-Distillation) Our framework involves sparse label-appearance encoding, inter-object relation adaptation and intra-object knowledge mapping to obtain the instructive knowledge. Compared with a classical teacher-based method FGFI, LGD not only performs better without requiring pretrained teacher but also with 51% lower training cost beyond inherent student learning.
  arXiv  Detail & Related papers  (2021-09-23T16:55:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.