Towards interpretable models for language proficiency assessment: Predicting the CEFR level of Estonian learner texts

• URL: http://arxiv.org/abs/2602.13102v1
• Date: Fri, 13 Feb 2026 17:06:17 GMT
• Title: Towards interpretable models for language proficiency assessment: Predicting the CEFR level of Estonian learner texts
• Authors: Kais Allkivi,
• Abstract summary: This study aimed to classify Estonian proficiency examination writings (levels A2-C1)<n>Various linguistic properties of the training data were analyzed to identify relevant proficiency predictors.<n>The results have been implemented in the writing evaluation module of an Estonian open-source language learning environment.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Using NLP to analyze authentic learner language helps to build automated assessment and feedback tools. It also offers new and extensive insights into the development of second language production. However, there is a lack of research explicitly combining these aspects. This study aimed to classify Estonian proficiency examination writings (levels A2-C1), assuming that careful feature selection can lead to more explainable and generalizable machine learning models for language testing. Various linguistic properties of the training data were analyzed to identify relevant proficiency predictors associated with increasing complexity and correctness, rather than the writing task. Such lexical, morphological, surface, and error features were used to train classification models, which were compared to models that also allowed for other features. The pre-selected features yielded a similar test accuracy but reduced variation in the classification of different text types. The best classifiers achieved an accuracy of around 0.9. Additional evaluation on an earlier exam sample revealed that the writings have become more complex over a 7-10-year period, while accuracy still reached 0.8 with some feature sets. The results have been implemented in the writing evaluation module of an Estonian open-source language learning environment.

Related papers

• Avaliação de eficiência na leitura: uma abordagem baseada em PLN [0.0]
  This study proposes an automated evaluation model for the cloze test in Brazilian Portuguese.<n>The integrated method demonstrated its effectiveness, achieving a high correlation with human evaluation.
  arXiv  Detail & Related papers  (2025-08-18T02:21:12Z)
• data2lang2vec: Data Driven Typological Features Completion [8.28573483085828]
  We introduce a multi-lingual Part-of-Speech (POS) tagger, achieving over 70% accuracy across 1,749 languages. We also introduce a more realistic evaluation setup, focusing on likely to be missing typology features.
  arXiv  Detail & Related papers  (2024-09-25T21:32:57Z)
• Language Models for Text Classification: Is In-Context Learning Enough? [54.869097980761595]
  Recent foundational language models have shown state-of-the-art performance in many NLP tasks in zero- and few-shot settings. An advantage of these models over more standard approaches is the ability to understand instructions written in natural language (prompts) This makes them suitable for addressing text classification problems for domains with limited amounts of annotated instances.
  arXiv  Detail & Related papers  (2024-03-26T12:47:39Z)
• Disco-Bench: A Discourse-Aware Evaluation Benchmark for Language Modelling [70.23876429382969]
  We propose a benchmark that can evaluate intra-sentence discourse properties across a diverse set of NLP tasks. Disco-Bench consists of 9 document-level testsets in the literature domain, which contain rich discourse phenomena. For linguistic analysis, we also design a diagnostic test suite that can examine whether the target models learn discourse knowledge.
  arXiv  Detail & Related papers  (2023-07-16T15:18:25Z)
• A Unified Neural Network Model for Readability Assessment with Feature Projection and Length-Balanced Loss [17.213602354715956]
  We propose a BERT-based model with feature projection and length-balanced loss for readability assessment. Our model achieves state-of-the-art performances on two English benchmark datasets and one dataset of Chinese textbooks.
  arXiv  Detail & Related papers  (2022-10-19T05:33:27Z)
• A Transfer Learning Based Model for Text Readability Assessment in German [4.550811027560416]
  We propose a new model for text complexity assessment for German text based on transfer learning. Best model is based on the BERT pre-trained language model achieved the Root Mean Square Error (RMSE) of 0.483.
  arXiv  Detail & Related papers  (2022-07-13T15:15:44Z)
• AES Systems Are Both Overstable And Oversensitive: Explaining Why And Proposing Defenses [66.49753193098356]
  We investigate the reason behind the surprising adversarial brittleness of scoring models. Our results indicate that autoscoring models, despite getting trained as "end-to-end" models, behave like bag-of-words models. We propose detection-based protection models that can detect oversensitivity and overstability causing samples with high accuracies.
  arXiv  Detail & Related papers  (2021-09-24T03:49:38Z)
• Improving Cross-Lingual Reading Comprehension with Self-Training [62.73937175625953]
  Current state-of-the-art models even surpass human performance on several benchmarks. Previous works have revealed the abilities of pre-trained multilingual models for zero-shot cross-lingual reading comprehension. This paper further utilized unlabeled data to improve the performance.
  arXiv  Detail & Related papers  (2021-05-08T08:04:30Z)
• TextFlint: Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing [73.16475763422446]
  We propose a multilingual robustness evaluation platform for NLP tasks (TextFlint) It incorporates universal text transformation, task-specific transformation, adversarial attack, subpopulation, and their combinations to provide comprehensive robustness analysis. TextFlint generates complete analytical reports as well as targeted augmented data to address the shortcomings of the model's robustness.
  arXiv  Detail & Related papers  (2021-03-21T17:20:38Z)
• Comparison of Interactive Knowledge Base Spelling Correction Models for Low-Resource Languages [81.90356787324481]
  Spelling normalization for low resource languages is a challenging task because the patterns are hard to predict. This work shows a comparison of a neural model and character language models with varying amounts on target language data. Our usage scenario is interactive correction with nearly zero amounts of training examples, improving models as more data is collected.
  arXiv  Detail & Related papers  (2020-10-20T17:31:07Z)
• Linguistic Features for Readability Assessment [0.0]
  It is unknown whether augmenting deep learning models with linguistically motivated features would improve performance further. We find that, given sufficient training data, augmenting deep learning models with linguistically motivated features does not improve state-of-the-art performance. Our results provide preliminary evidence for the hypothesis that the state-of-the-art deep learning models represent linguistic features of the text related to readability.
  arXiv  Detail & Related papers  (2020-05-30T22:14:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.