Clause Encounters of the Third Kind: Can LLMs Replace Language Teachers?

• URL: http://arxiv.org/abs/2608.16286v1
• Date: Mon, 17 Aug 2026 08:58:30 GMT
• Title: Clause Encounters of the Third Kind: Can LLMs Replace Language Teachers?
• Abstract summary: This paper examines whether state-of-the-art LLMs can deliver the kind of corrective feedback and methodological explanations that language learners need.<n>The evaluation employs automated metrics (GLEU, BERTScore) but also human expert judgments to capture dimensions that purely computational measures miss.<n>While models demonstrate impressive surface-level correction abilities, their explanations often lack the terminological and domain knowledge that effective language teaching requires.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While various organizations now actively encourage LLM use in classrooms, we still lack rigorous, systematic evaluations of how well these models actually perform the fundamental tasks of language pedagogy. This paper examines whether state-of-the-art LLMs can deliver the kind of corrective feedback and methodological explanations that language learners need. The study tests multiple large language models on their ability to identify, correct, and explain common learner mistakes in English, by systematically varying model parameters to investigate how these technical adjustments affect output quality, pedagogical clarity, and consistency, along with using retrieval-augmented generation to query methodological data. The evaluation employs automated metrics (GLEU, BERTScore) but also human expert judgments to capture dimensions that purely computational measures miss: linguistic nuance, cultural sensitivity, and instructional appropriateness. While models demonstrate impressive surface-level correction abilities, their explanations often lack the terminological and domain knowledge that effective language teaching requires, suggesting that current enthusiasm for AI-assisted language learning may be outpacing our understanding of these systems' actual pedagogical competence.

Related papers

• SLATE: Are AI-Generated Slides Educationally Effective? A Benchmark for Language Teaching Quality and Learner Knowledge Acquisition [12.243304149666258]
  We introduce SLATE, the first benchmark that evaluates AI-generated language teaching slides through instructional effectiveness and learner knowledge acquisition.<n>SLATE transforms linguistics olympiad puzzles from low-resource languages with negligible web presence into 90 standardized instructional units.<n>Our results show that content validity exhibits a weak association with learning gain, while pedagogical design exhibits a robust positive association.
  arXiv  Detail & Related papers  (2026-09-05T18:14:23Z)
• Large Language Models as Students Who Think Aloud: Overly Coherent, Verbose, and Confident [0.8564319625930894]
  Large language models (LLMs) are increasingly embedded in AI-based tutoring systems. Can they faithfully model novice reasoning and metacognitive judgments?<n>We evaluate LLMs as novices using 630 think-aloud utterances from chemistry tutoring problems with problem-solving logs of student hint use, attempts, and problem context.<n>We compare LLM-generated reasoning to human learner utterances under minimal and extended contextual prompting, and assess the models' ability to predict step-level learner success.
  arXiv  Detail & Related papers  (2026-02-01T04:46:38Z)
• FEANEL: A Benchmark for Fine-Grained Error Analysis in K-12 English Writing [68.23874413455594]
  We present the Fine-grained Error ANalysis for English learners (FEANEL) Benchmark.<n>The benchmark comprises 1,000 essays written by elementary and secondary school students.<n>Each error is annotated by language education experts and categorized by type, severity, and explanatory feedback, using a part-of-speech-based taxonomy they co-developed.
  arXiv  Detail & Related papers  (2025-11-28T05:17:45Z)
• Speech-IFEval: Evaluating Instruction-Following and Quantifying Catastrophic Forgetting in Speech-Aware Language Models [49.1574468325115]
  We introduce Speech-IFeval, an evaluation framework designed to assess instruction-following capabilities.<n>Recent SLMs integrate speech perception with large language models (LLMs), often degrading textual capabilities due to speech-centric training.<n>Our findings show that most SLMs struggle with even basic instructions, performing far worse than text-based LLMs.
  arXiv  Detail & Related papers  (2025-05-25T08:37:55Z)
• CPG-EVAL: A Multi-Tiered Benchmark for Evaluating the Chinese Pedagogical Grammar Competence of Large Language Models [6.0020878662404975]
  This paper introduces the first benchmark specifically designed to evaluate LLMs' knowledge of pedagogical grammar within the context of foreign language instruction.<n>The benchmark comprises five tasks designed to assess grammar recognition, fine-grained grammatical distinction, categorical discrimination, and resistance to linguistic interference.
  arXiv  Detail & Related papers  (2025-04-17T18:01:50Z)
• LLM-as-a-Judge & Reward Model: What They Can and Cannot Do [2.2469442203227863]
  We conduct a comprehensive analysis of automated evaluators, reporting several key findings on their behavior. We discover that English evaluation capabilities significantly influence language-specific evaluation capabilities, enabling evaluators trained in English to easily transfer their skills to other languages. We find that state-of-the-art evaluators struggle with challenging prompts, in either English or Korean, underscoring their limitations in assessing or generating complex reasoning questions.
  arXiv  Detail & Related papers  (2024-09-17T14:40:02Z)
• CourseAssist: Pedagogically Appropriate AI Tutor for Computer Science Education [1.052788652996288]
  This poster introduces CourseAssist, a novel LLM-based tutoring system tailored for computer science education. Unlike generic LLM systems, CourseAssist uses retrieval-augmented generation, user intent classification, and question decomposition to align AI responses with specific course materials and learning objectives.
  arXiv  Detail & Related papers  (2024-05-01T20:43:06Z)
• Teacher Perception of Automatically Extracted Grammar Concepts for L2 Language Learning [66.79173000135717]
  We apply this work to teaching two Indian languages, Kannada and Marathi, which do not have well-developed resources for second language learning. We extract descriptions from a natural text corpus that answer questions about morphosyntax (learning of word order, agreement, case marking, or word formation) and semantics (learning of vocabulary). We enlist the help of language educators from schools in North America to perform a manual evaluation, who find the materials have potential to be used for their lesson preparation and learner evaluation.
  arXiv  Detail & Related papers  (2023-10-27T18:17:29Z)
• Spoken Language Intelligence of Large Language Models for Language Learning [3.1964044595140217]
  We focus on evaluating the efficacy of large language models (LLMs) in the realm of education.<n>We introduce a new multiple-choice question dataset to evaluate the effectiveness of LLMs in the aforementioned scenarios.<n>We also investigate the influence of various prompting techniques such as zero- and few-shot method.<n>We find that models of different sizes have good understanding of concepts in phonetics, phonology, and second language acquisition, but show limitations in reasoning for real-world problems.
  arXiv  Detail & Related papers  (2023-08-28T12:47:41Z)
• Evaluating Language Models for Mathematics through Interactions [116.67206980096513]
  We introduce CheckMate, a prototype platform for humans to interact with and evaluate large language models (LLMs) We conduct a study with CheckMate to evaluate three language models (InstructGPT, ChatGPT, and GPT-4) as assistants in proving undergraduate-level mathematics. We derive a taxonomy of human behaviours and uncover that despite a generally positive correlation, there are notable instances of divergence between correctness and perceived helpfulness.
  arXiv  Detail & Related papers  (2023-06-02T17:12:25Z)
• Shortcut Learning of Large Language Models in Natural Language Understanding [119.45683008451698]
  Large language models (LLMs) have achieved state-of-the-art performance on a series of natural language understanding tasks. They might rely on dataset bias and artifacts as shortcuts for prediction. This has significantly affected their generalizability and adversarial robustness.
  arXiv  Detail & Related papers  (2022-08-25T03:51:39Z)
• Teacher Perception of Automatically Extracted Grammar Concepts for L2 Language Learning [91.49622922938681]
  We present an automatic framework that automatically discovers and visualizing descriptions of different aspects of grammar. Specifically, we extract descriptions from a natural text corpus that answer questions about morphosyntax and semantics. We apply this method for teaching the Indian languages, Kannada and Marathi, which, unlike English, do not have well-developed pedagogical resources.
  arXiv  Detail & Related papers  (2022-06-10T14:52:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.