Can LLMs Control Readability? A Multi-Dimensional Evaluation Framework for CEFR-Controlled Arabic Generation

• URL: http://arxiv.org/abs/2606.21981v1
• Date: Sat, 20 Jun 2026 10:30:33 GMT
• Title: Can LLMs Control Readability? A Multi-Dimensional Evaluation Framework for CEFR-Controlled Arabic Generation
• Abstract summary: Large Language Models (LLMs) can generate fluent Arabic text, but their ability to reliably control readability levels remains unclear.<n>We propose a multi-dimensional evaluation framework for Common European Framework of Reference for Language (CEFR)-controlled Arabic text generation.<n>Our framework integrates controlled prompting, automatic readability prediction using a validated Taha-19 model, lexical constraint validation, and syntactic complexity profiling.
• Score: 2.831104808316866
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While Large Language Models (LLMs) can generate fluent Arabic text, their ability to reliably control readability levels remains unclear. We propose a multi-dimensional evaluation framework for Common European Framework of Reference for Language (CEFR)-controlled Arabic text generation, assessing whether instruction-following LLMs can serve as reliable generators for adaptive language learning. Our framework integrates controlled prompting, automatic readability prediction using a validated Taha-19 model, lexical constraint validation, and syntactic complexity profiling. Results show that structured prompting substantially improves CEFR alignment. In particular, CEFR-guided prompting with lexical constraints achieves the highest conformity to reference linguistic profiles (0.91 cosine similarity) and near-perfect agreement with predicted readability levels (0.99), while unconstrained prompting exhibits weak control. These findings establish an empirical foundation for integrating readability-aware Arabic text generation into adaptive educational systems.

Related papers

• Learning and Enforcing Context-Sensitive Control for LLMs [48.58004364546273]
  We introduce a framework that automatically learns context-sensitive constraints from Large Language Models interactions.<n>Experiments demonstrate that our method enables even small LLMs (1B parameters) to learn and generate with perfect constraint adherence.
  arXiv  Detail & Related papers  (2026-04-12T14:50:03Z)
• Plausibility as Commonsense Reasoning: Humans Succeed, Large Language Models Do not [0.0]
  We test whether large language models integrate world knowledge with syntactic structure in a human-like, structure-sensitive way during ambiguity resolution.<n>We construct ambiguous items that keep the syntactic configuration fixed and ensure both parses remain pragmatically possible.<n>In a speeded forced-choice comprehension experiment, humans show a large, correctly directed plausibility effect.<n>Across models, plausibility-driven shifts are weak, unstable, or reversed.
  arXiv  Detail & Related papers  (2026-04-06T16:27:06Z)
• Classifying German Language Proficiency Levels Using Large Language Models [0.24683296459020942]
  This paper investigates the use of Large Language Models (LLMs) for automatically classifying German texts into different proficiency levels.<n>To support robust training and evaluation, we construct a diverse dataset by combining multiple existing CEFR-annotated corpora with synthetic data.<n>Our results show a consistent performance improvement over prior methods, highlighting the potential of LLMs for reliable and scalable CEFR classification.
  arXiv  Detail & Related papers  (2025-12-06T16:15:45Z)
• LexInstructEval: Lexical Instruction Following Evaluation for Large Language Models [7.921987175359344]
  LexInstructEval is a new benchmark and evaluation framework for fine-grained lexical instruction following.<n>Our framework is built upon a formal, rule-based grammar that deconstructs complex instructions into a canonical Procedure, Relation, Value> triplet.<n>This grammar enables the systematic generation of a diverse dataset through a multi-stage, human-in-the-loop pipeline.
  arXiv  Detail & Related papers  (2025-11-13T08:04:30Z)
• MoRE-LLM: Mixture of Rule Experts Guided by a Large Language Model [54.14155564592936]
  We propose a Mixture of Rule Experts guided by a Large Language Model (MoRE-LLM)<n>MoRE-LLM steers the discovery of local rule-based surrogates during training and their utilization for the classification task.<n>LLM is responsible for enhancing the domain knowledge alignment of the rules by correcting and contextualizing them.
  arXiv  Detail & Related papers  (2025-03-26T11:09:21Z)
• CATER: Leveraging LLM to Pioneer a Multidimensional, Reference-Independent Paradigm in Translation Quality Evaluation [0.0]
  Comprehensive AI-assisted Translation Edit Ratio (CATER) is a novel framework for evaluating machine translation (MT) quality.<n>Uses large language models (LLMs) via a carefully designed prompt-based protocol.
  arXiv  Detail & Related papers  (2024-12-15T17:45:34Z)
• ICLEval: Evaluating In-Context Learning Ability of Large Language Models [68.7494310749199]
  In-Context Learning (ICL) is a critical capability of Large Language Models (LLMs) as it empowers them to comprehend and reason across interconnected inputs.<n>Existing evaluation frameworks primarily focus on language abilities and knowledge, often overlooking the assessment of ICL ability.<n>We introduce the ICLEval benchmark to evaluate the ICL abilities of LLMs, which encompasses two key sub-abilities: exact copying and rule learning.
  arXiv  Detail & Related papers  (2024-06-21T08:06:10Z)
• CELA: Cost-Efficient Language Model Alignment for CTR Prediction [70.65910069412944]
  Click-Through Rate (CTR) prediction holds a paramount position in recommender systems.<n>Recent efforts have sought to mitigate these challenges by integrating Pre-trained Language Models (PLMs)<n>We propose textbfCost-textbfEfficient textbfLanguage Model textbfAlignment (textbfCELA) for CTR prediction.
  arXiv  Detail & Related papers  (2024-05-17T07:43:25Z)
• Embracing Language Inclusivity and Diversity in CLIP through Continual Language Learning [58.92843729869586]
  Vision-language pre-trained models (VL-PTMs) have advanced multimodal research in recent years, but their mastery in a few languages like English restricts their applicability in broader communities. We propose to extend VL-PTMs' language capacity by continual language learning (CLL), where a model needs to update its linguistic knowledge incrementally without suffering from catastrophic forgetting (CF) We construct a CLL benchmark covering 36 languages based on MSCOCO and XM3600 datasets and then evaluate multilingual image-text retrieval performance.
  arXiv  Detail & Related papers  (2024-01-30T17:14:05Z)
• Evaluating, Understanding, and Improving Constrained Text Generation for Large Language Models [49.74036826946397]
  This study investigates constrained text generation for large language models (LLMs) Our research mainly focuses on mainstream open-source LLMs, categorizing constraints into lexical, structural, and relation-based types. Results illuminate LLMs' capacity and deficiency to incorporate constraints and provide insights for future developments in constrained text generation.
  arXiv  Detail & Related papers  (2023-10-25T03:58:49Z)
• Flesch or Fumble? Evaluating Readability Standard Alignment of Instruction-Tuned Language Models [4.867923281108005]
  We select a diverse set of open and closed-source instruction-tuned language models and investigate their performances in writing story completions and simplifying narratives. Our findings provide empirical proof of how globally recognized models like ChatGPT may be considered less effective and may require more refined prompts for these generative tasks.
  arXiv  Detail & Related papers  (2023-09-11T13:50:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.