Exploiting the English Grammar Profile for L2 grammatical analysis with LLMs

• URL: http://arxiv.org/abs/2603.17171v1
• Date: Tue, 17 Mar 2026 22:06:00 GMT
• Title: Exploiting the English Grammar Profile for L2 grammatical analysis with LLMs
• Abstract summary: The English Grammar Profile (EGP) is a taxonomy of grammatical constructs mapped to the proficiency levels of the Common European Framework of Reference (CEFR)<n>We propose a novel framework leveraging the EGP to detect learners' attempts at grammatical constructs and classify them as successful or unsuccessful.
• Score: 3.027026792589421
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Evaluating the grammatical competence of second language (L2) learners is essential both for providing targeted feedback and for assessing proficiency. To achieve this, we propose a novel framework leveraging the English Grammar Profile (EGP), a taxonomy of grammatical constructs mapped to the proficiency levels of the Common European Framework of Reference (CEFR), to detect learners' attempts at grammatical constructs and classify them as successful or unsuccessful. This detection can then be used to provide fine-grained feedback. Moreover, the grammatical constructs are used as predictors of proficiency assessment by using automatically detected attempts as predictors of holistic CEFR proficiency. For the selection of grammatical constructs derived from the EGP, rule-based and LLM-based classifiers are compared. We show that LLMs outperform rule-based methods on semantically and pragmatically nuanced constructs, while rule-based approaches remain competitive for constructs that rely purely on morphological or syntactic features and do not require semantic interpretation. For proficiency assessment, we evaluate both rule-based and hybrid pipelines and show that a hybrid approach combining a rule-based pre-filter with an LLM consistently yields the strongest performance. Since our framework operates on pairs of original learner sentences and their corrected counterparts, we also evaluate a fully automated pipeline using automatic grammatical error correction. This pipeline closely approaches the performance of semi-automated systems based on manual corrections, particularly for the detection of successful attempts at grammatical constructs. Overall, our framework emphasises learners' successful attempts in addition to unsuccessful ones, enabling positive, formative feedback and providing actionable insights into grammatical development.

Related papers

• How Useful are LLMs for Grammar Engineering? Cantonese ParGram Resources and Controlled Experimental Evaluation with English Baselines [0.0]
  This paper presents new Cantonese ParGram resources and evaluates LLMs for knowledge-driven grammar engineering.<n>We investigate whether OpenAI's.<n>gpt-oss-120b and GPT-5.4 can generate machine-processable grammars.<n>from sentences and target formal structures under.<n>systematically varied prompting conditions.
  arXiv  Detail & Related papers  (2026-08-24T16:21:32Z)
• The Regularizing Power of Language-Training Deepfake Detectors [57.529452351298495]
  Deepfake detection generally suffers from overfitting to low-level domain-specific artifacts.<n>We use a dual-encoder architecture, pairing a frozen specialist detector with a LoRA-tuned MLLM encoder.<n>We employ a reinforcement learning stage that encourages the model to generate descriptive reasoning before classifying.
  arXiv  Detail & Related papers  (2026-05-29T12:01:17Z)
• From Prompting to Preference Optimization: A Comparative Study of LLM-based Automated Essay Scoring [2.3843476850953995]
  Large language models (LLMs) have recently reshaped Automated Essay Scoring (AES)<n>We present a comprehensive comparison of major LLM-based AES paradigms on Writing Task2.<n>Our results reveal clear accuracy-cost-robustness trade-offs across methods, the best configuration, integrating k-SFT and RAG, achieves the strongest overall results with F1-Score 93%.
  arXiv  Detail & Related papers  (2026-03-06T15:59:32Z)
• SAGE:Specification-Aware Grammar Extraction for Automated Test Case Generation with LLMs [8.576669309128382]
  We explore the use of open-source large language models (LLMs) to induce Context-Free Grammars with Counters (CCFGs)<n>CCFGs represent specifications with logical constraints by storing and reusing counter values during derivation.<n>Our approach achieves stronger generalization and outperforms 17 open and closed-source LLMs in both grammar quality and test effectiveness.
  arXiv  Detail & Related papers  (2025-06-04T08:44:25Z)
• MoRE-LLM: Mixture of Rule Experts Guided by a Large Language Model [54.14155564592936]
  We propose a Mixture of Rule Experts guided by a Large Language Model (MoRE-LLM)<n>MoRE-LLM steers the discovery of local rule-based surrogates during training and their utilization for the classification task.<n>LLM is responsible for enhancing the domain knowledge alignment of the rules by correcting and contextualizing them.
  arXiv  Detail & Related papers  (2025-03-26T11:09:21Z)
• Assessing the validity of new paradigmatic complexity measures as criterial features for proficiency in L2 writings in English [1.465540676497032]
  This article addresses Second Language (L2) writing development through an investigation of new grammatical and structural complexity metrics.<n>We explore the paradigmatic production in learner English by linking language functions to specific grammatical paradigms.
  arXiv  Detail & Related papers  (2025-03-13T10:01:07Z)
• Analysis of LLM as a grammatical feature tagger for African American English [0.6927055673104935]
  African American English (AAE) presents unique challenges in natural language processing (NLP)<n>This research systematically compares the performance of available NLP models.<n>This study highlights the necessity for improved model training and architectural adjustments to better accommodate AAE's unique linguistic characteristics.
  arXiv  Detail & Related papers  (2025-02-09T19:46:33Z)
• Preference Alignment Improves Language Model-Based TTS [76.70693823683091]
  preference alignment algorithms adjust LMs to align with the preferences of reward models, enhancing the desirability of the generated content. With a 1.15B parameter LM-based TTS model, we demonstrate that preference alignment consistently improves intelligibility, speaker similarity, and proxy subjective evaluation scores.
  arXiv  Detail & Related papers  (2024-09-19T01:58:19Z)
• DECIDER: A Dual-System Rule-Controllable Decoding Framework for Language Generation [57.07295906718989]
  Constrained decoding approaches aim to control the meaning or style of text generated by pre-trained large language (Ms also PLMs) for various tasks at inference time.<n>These methods often guide plausible continuations by greedily and explicitly selecting targets.<n>Inspired by cognitive dual-process theory, we propose a novel decoding framework DECIDER.
  arXiv  Detail & Related papers  (2024-03-04T11:49:08Z)
• Large Language Models can Contrastively Refine their Generation for Better Sentence Representation Learning [57.74233319453229]
  Large language models (LLMs) have emerged as a groundbreaking technology and their unparalleled text generation capabilities have sparked interest in their application to the fundamental sentence representation learning task. We propose MultiCSR, a multi-level contrastive sentence representation learning framework that decomposes the process of prompting LLMs to generate a corpus. Our experiments reveal that MultiCSR enables a less advanced LLM to surpass the performance of ChatGPT, while applying it to ChatGPT achieves better state-of-the-art results.
  arXiv  Detail & Related papers  (2023-10-17T03:21:43Z)
• Structural Pre-training for Dialogue Comprehension [51.215629336320305]
  We present SPIDER, Structural Pre-traIned DialoguE Reader, to capture dialogue exclusive features. To simulate the dialogue-like features, we propose two training objectives in addition to the original LM objectives. Experimental results on widely used dialogue benchmarks verify the effectiveness of the newly introduced self-supervised tasks.
  arXiv  Detail & Related papers  (2021-05-23T15:16:54Z)
• SLM: Learning a Discourse Language Representation with Sentence Unshuffling [53.42814722621715]
  We introduce Sentence-level Language Modeling, a new pre-training objective for learning a discourse language representation. We show that this feature of our model improves the performance of the original BERT by large margins.
  arXiv  Detail & Related papers  (2020-10-30T13:33:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.