Automated Scoring of Arabic Text Using Large Language Models: A Literature Review

• URL: http://arxiv.org/abs/2606.09830v2
• Date: Wed, 10 Jun 2026 13:54:49 GMT
• Title: Automated Scoring of Arabic Text Using Large Language Models: A Literature Review
• Abstract summary: This work focuses on the automated evaluation of Arabic texts, focusing on both short answer grading (ASAG) and essay scoring (AES)<n>We introduce a structured taxonomy comprising five dimensions: application domain, feedback generation capability, LLM architecture deployed, alignment with competency referential frameworks, and prompt engineering strategy.<n>The findings highlight the need for sustained and pedagogically grounded research efforts in Arabic ATS, given its significance for improving educational quality across Arabic-speaking communities.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: In modern educational systems, Automatic Text Scoring (ATS) plays a central role by enabling scalable and consistent evaluation of learner responses without human intervention. Recently, the increased accessibility of LLMs and Arabic-specific datasets has sparked renewed interest in this area. In this work, we investigate LLM-Based approaches for the automated evaluation of Arabic texts, focusing on both short answer grading (ASAG) and essay scoring (AES). We further introduce a structured taxonomy comprising five dimensions: application domain, feedback generation capability, LLM architecture deployed, alignment with competency referential frameworks, and prompt engineering strategy. By applying this taxonomy, we conduct a comparative analysis of existing studies, examining their methodological approaches, datasets, evaluation metrics, and reported performance. The findings highlight the need for sustained and pedagogically grounded research efforts in Arabic ATS, given its significance for improving educational quality across Arabic-speaking communities.

Related papers

• Structured Prompting for Arabic Essay Proficiency: A Trait-Centric Evaluation Approach [3.0802183987532725]
  This paper presents a novel prompt engineering framework for trait specific Automatic Essay Scoring (AES) in Arabic.<n>We introduce a three-tier prompting strategy that guides LLMs in evaluating distinct language proficiency traits such as organization, vocabulary, development, and style.<n> Experimental results show that Fanar-1-9B-Instruct achieves the highest trait level agreement in both zero and few-shot prompting.
  arXiv  Detail & Related papers  (2026-03-20T06:05:04Z)
• LLM-based Automated Grading with Human-in-the-Loop [32.14015215819979]
  Large language models (LLMs) are increasingly being used for automatic short answer grading (ASAG)<n>In this work, we explore the potential of LLMs in ASAG tasks by leveraging their interactive capabilities through a human-in-the-loop (HITL) approach.<n>Our proposed framework, GradeHITL, utilizes the generative properties of LLMs to pose questions to human experts, incorporating their insights to refine grading rubrics dynamically.
  arXiv  Detail & Related papers  (2025-04-07T16:23:07Z)
• Evaluating LLM-based Agents for Multi-Turn Conversations: A Survey [64.08485471150486]
  This survey examines evaluation methods for large language model (LLM)-based agents in multi-turn conversational settings.<n>We systematically reviewed nearly 250 scholarly sources, capturing the state of the art from various venues of publication.
  arXiv  Detail & Related papers  (2025-03-28T14:08:40Z)
• How well can LLMs Grade Essays in Arabic? [3.101490720236325]
  This research assesses the effectiveness of large language models (LLMs) in the task of Arabic automated essay scoring (AES) using the AR-AES dataset.<n>It explores various evaluation methodologies, including zero-shot, few-shot in-context learning, and fine-tuning.<n>A mixed-language prompting strategy, integrating English prompts with Arabic content, was implemented to improve model comprehension and performance.
  arXiv  Detail & Related papers  (2025-01-27T21:30:02Z)
• Large Language Model for Qualitative Research -- A Systematic Mapping Study [3.302912592091359]
  Large Language Models (LLMs), powered by advanced generative AI, have emerged as transformative tools.<n>This study systematically maps the literature on the use of LLMs for qualitative research.<n>Findings reveal that LLMs are utilized across diverse fields, demonstrating the potential to automate processes.
  arXiv  Detail & Related papers  (2024-11-18T21:28:00Z)
• A Survey of Small Language Models [104.80308007044634]
  Small Language Models (SLMs) have become increasingly important due to their efficiency and performance to perform various language tasks with minimal computational resources. We present a comprehensive survey on SLMs, focusing on their architectures, training techniques, and model compression techniques.
  arXiv  Detail & Related papers  (2024-10-25T23:52:28Z)
• Are Large Language Models Good Classifiers? A Study on Edit Intent Classification in Scientific Document Revisions [62.12545440385489]
  Large language models (LLMs) have brought substantial advancements in text generation, but their potential for enhancing classification tasks remains underexplored. We propose a framework for thoroughly investigating fine-tuning LLMs for classification, including both generation- and encoding-based approaches. We instantiate this framework in edit intent classification (EIC), a challenging and underexplored classification task.
  arXiv  Detail & Related papers  (2024-10-02T20:48:28Z)
• Disce aut Deficere: Evaluating LLMs Proficiency on the INVALSI Italian Benchmark [12.729687989535359]
  evaluating Large Language Models (LLMs) in languages other than English is crucial for ensuring their linguistic versatility, cultural relevance, and applicability in diverse global contexts. We tackle this challenge by introducing a structured benchmark using the INVALSI tests, a set of well-established assessments designed to measure educational competencies across Italy.
  arXiv  Detail & Related papers  (2024-06-25T13:20:08Z)
• A Literature Review of Literature Reviews in Pattern Analysis and Machine Intelligence [51.26815896167173]
  We present a comprehensive tertiary analysis of PAMI reviews along three complementary dimensions.<n>Our analyses reveal distinctive organizational patterns as well as persistent gaps in current review practices.<n>Finally, our evaluation of state-of-the-art AI-generated reviews indicates encouraging advances in coherence and organization.
  arXiv  Detail & Related papers  (2024-02-20T11:28:50Z)
• Leveraging Large Language Models for NLG Evaluation: Advances and Challenges [57.88520765782177]
  Large Language Models (LLMs) have opened new avenues for assessing generated content quality, e.g., coherence, creativity, and context relevance. We propose a coherent taxonomy for organizing existing LLM-based evaluation metrics, offering a structured framework to understand and compare these methods. By discussing unresolved challenges, including bias, robustness, domain-specificity, and unified evaluation, this paper seeks to offer insights to researchers and advocate for fairer and more advanced NLG evaluation techniques.
  arXiv  Detail & Related papers  (2024-01-13T15:59:09Z)
• Investigating Fairness Disparities in Peer Review: A Language Model Enhanced Approach [77.61131357420201]
  We conduct a thorough and rigorous study on fairness disparities in peer review with the help of large language models (LMs) We collect, assemble, and maintain a comprehensive relational database for the International Conference on Learning Representations (ICLR) conference from 2017 to date. We postulate and study fairness disparities on multiple protective attributes of interest, including author gender, geography, author, and institutional prestige.
  arXiv  Detail & Related papers  (2022-11-07T16:19:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.