EduIllustrate: Towards Scalable Automated Generation Of Multimodal Educational Content

• URL: http://arxiv.org/abs/2604.05005v2
• Date: Sat, 11 Apr 2026 07:08:48 GMT
• Title: EduIllustrate: Towards Scalable Automated Generation Of Multimodal Educational Content
• Abstract summary: We present EduIllustrate, a benchmark for evaluating large language models.<n>The benchmark comprises 230 problems spanning five subjects and three grade levels.<n>We show that Gemini 3.0 Pro Preview leads at 87.8%, while Kimi-K2.5 achieves the best cost-efficiency.
• Score: 19.131221541276332
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models are increasingly used as educational assistants, yet evaluation of their educational capabilities remains concentrated on question-answering and tutoring tasks. A critical gap exists for multimedia instructional content generation -- the ability to produce coherent, diagram-rich explanations that combine geometrically accurate visuals with step-by-step reasoning. We present EduIllustrate, a benchmark for evaluating LLMs on interleaved text-diagram explanation generation for K-12 STEM problems. The benchmark comprises 230 problems spanning five subjects and three grade levels, a standardized generation protocol with sequential anchoring to enforce cross-diagram visual consistency, and an 8-dimension evaluation rubric grounded in multimedia learning theory covering both text and visual quality. Evaluation of ten LLMs reveals a wide performance spread: Gemini 3.0 Pro Preview leads at 87.8\%, while Kimi-K2.5 achieves the best cost-efficiency (80.8\% at \\$0.12/problem). Workflow ablation confirms sequential anchoring improves Visual Consistency by 13\% at 94\% lower cost. Human evaluation with 20 expert raters validates LLM-as-judge reliability for objective dimensions ($ρ\geq 0.83$) while revealing limitations on subjective visual assessment.

Related papers

• CURV: Enhancing Chart Understanding Through Curriculum Visual Grounded Reasoning [58.582602613053204]
  Chart question answering (CQA) requires multimodal large language models (MLLMs) to integrate visual comprehension with logical reasoning.<n>Current MLLMs lack intrinsic visual grounded reasoning capabilities, leading to inaccurate perception and reasoning disconnected from visual evidence.<n>We propose CURV, a curriculum learning framework that develops intrinsic visual reasoning capabilities by reformulating CQA as multi-step visual grounded reasoning.
  arXiv  Detail & Related papers  (2026-08-03T19:48:00Z)
• Exploring Agentic Workflows for Generating High Quality Math Visual Aids [0.8228126670075446]
  We introduce an agentic workflow that enables Large Language Models (LLMs) to evaluate the quality of generated visuals.<n>This self improvement loop aims to enhance the accuracy and educational appropriateness of AI generated diagrams.
  arXiv  Detail & Related papers  (2026-07-10T17:12:38Z)
• TeachObs: A Human-Validated Benchmark for Multimodal Teaching Observation and Model Evaluation [0.12079404108544865]
  We present textitTeachObs, a human-validated benchmark for multimodal teaching observation in classroom videos.<n>textitTeachObs includes 30 public lesson videos from eight countries divided into 5,158 fixed 15-second scenes.
  arXiv  Detail & Related papers  (2026-05-29T00:06:54Z)
• An Interpretable Closed-Loop Intelligent Tutoring System for Multimodal Affective Feedback in Asynchronous Presentation Training [0.0]
  The ITS maps multimodal inputs into evidence-based feedback that can be traced back to observable performance cues.<n>The system achieved rubric-aligned scoring with performance levels comparable to expert ratings.
  arXiv  Detail & Related papers  (2026-05-17T14:12:40Z)
• LecEval: An Automated Metric for Multimodal Knowledge Acquisition in Multimedia Learning [58.98865450345401]
  We introduce LecEval, an automated metric grounded in Mayer's Cognitive Theory of Multimedia Learning.<n>LecEval assesses effectiveness using four rubrics: Content Relevance (CR), Expressive Clarity (EC), Logical Structure (LS) and Audience Engagement (AE)<n>We curate a large-scale dataset of over 2,000 slides from more than 50 online course videos, annotated with fine-grained human ratings.
  arXiv  Detail & Related papers  (2025-05-04T12:06:47Z)
• VOILA: Evaluation of MLLMs For Perceptual Understanding and Analogical Reasoning [63.0285363282581]
  Multimodal Large Language Models (MLLMs) have become a powerful tool for integrating visual and textual information.<n>We introduce VOILA, a benchmark designed to evaluate MLLMs' perceptual understanding and abstract relational reasoning.<n>We reveal that current MLLMs struggle to comprehend inter-image relationships and exhibit limited capabilities in high-level relational reasoning.
  arXiv  Detail & Related papers  (2025-02-25T23:36:19Z)
• VLM2-Bench: A Closer Look at How Well VLMs Implicitly Link Explicit Matching Visual Cues [34.95077625513563]
  We introduce textbfVLM2-Bench, a benchmark designed to assess whether vision-language models can Visually Link Matching cues.<n> Comprehensive evaluation across twelve VLMs, along with further analysis of various language-side and vision-side prompting methods, leads to a total of eight key findings.<n>We identify critical challenges in models' ability to link visual cues, highlighting a significant performance gap.
  arXiv  Detail & Related papers  (2025-02-17T17:57:50Z)
• MMIE: Massive Multimodal Interleaved Comprehension Benchmark for Large Vision-Language Models [71.36392373876505]
  We introduce MMIE, a large-scale benchmark for evaluating interleaved multimodal comprehension and generation in Large Vision-Language Models (LVLMs)<n>MMIE comprises 20K meticulously curated multimodal queries, spanning 3 categories, 12 fields, and 102 subfields, including mathematics, coding, physics, literature, health, and arts.<n>It supports both interleaved inputs and outputs, offering a mix of multiple-choice and open-ended question formats to evaluate diverse competencies.
  arXiv  Detail & Related papers  (2024-10-14T04:15:00Z)
• VHELM: A Holistic Evaluation of Vision Language Models [75.88987277686914]
  We present the Holistic Evaluation of Vision Language Models (VHELM) VHELM aggregates various datasets to cover one or more of the 9 aspects: visual perception, knowledge, reasoning, bias, fairness, multilinguality, robustness, toxicity, and safety. Our framework is designed to be lightweight and automatic so that evaluation runs are cheap and fast.
  arXiv  Detail & Related papers  (2024-10-09T17:46:34Z)
• E.T. Bench: Towards Open-Ended Event-Level Video-Language Understanding [57.630136434038384]
  We introduce E.T. Bench (Event-Level & Time-Sensitive Video Understanding Benchmark), a large-scale benchmark for open-ended event-level video understanding. We extensively evaluated 8 Image-LLMs and 12 Video-LLMs on our benchmark, and the results reveal that state-of-the-art models for coarse-level (video-level) understanding struggle to solve our fine-grained tasks. Our simple but effective solution demonstrates superior performance in multiple scenarios.
  arXiv  Detail & Related papers  (2024-09-26T17:53:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.