ArguLens: An Open-Source System for Automated Essay Scoring and Label-Aware Feedback Generation

• URL: http://arxiv.org/abs/2608.17356v1
• Date: Tue, 18 Aug 2026 04:23:50 GMT
• Title: ArguLens: An Open-Source System for Automated Essay Scoring and Label-Aware Feedback Generation
• Abstract summary: ArguLens is an opensource system that decomposes automated essay scoring (AES) into three decoupled components.<n>A label-aware feedback generator serves through vLLM with a Qwen2.5-14BInstruct backbone.<n>A Gradio web UI exposes pluggable inference backends and supports single-essay and batch scoring.
• Score: 9.677955445684686
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Most automated essay scoring (AES) systems output a single holistic score without interpretable evidence and rely on closed APIs that introduce data privacy and cost barriers. We present ArguLens, an opensource, locally deployable system that decomposes AES into three decoupled components: a discourse-move classifier (Qwen2.5-7B-Instruct fine-tuned with LoRA on PERSUADE 2.0), a grade-independent LightGBM scorer over 31 linguistic and discourse features, and a label-aware feedback generator served through vLLM with a Qwen2.5-14BInstruct backbone. A Gradio web UI exposes pluggable inference backends and supports single-essay and batch scoring with downloadable per-essay breakdowns. On an essaydisjoint PERSUADE 2.0 test split, the logitprobe classifier achieves 82.6% accuracy and 0.727 macro-F1; under prompt-grouped 5-fold cross-validation the scorer reaches a mean QWK of 0.813 under an oracle discoursefeature protocol, and an ablation shows that adding gold discourse annotations yields an increment of +0.055 QWK over the lexical+syntactic configuration (paired t-test, p = 0.010). This is a component-level diagnostic rather than an end-to-end classifier-to-scorer result. The feedback generator ships with a structured evaluation protocol; its human-rater study is left to future work. The system is released under Apache 2.0 at https://github.com/wwrwbs/AI_AWE.

Related papers

• Staged Linguistic Seeding: Grounded Query Expansion for Verified-Unit QA in AI Contact Centers [1.1983198417783958]
  We deploy a system that answers only from a closed set of verified QA units.<n>One methodology is reused across both domains, so inference stays a single retrieval pass with no query-time generation.<n>We report this as an application study, including negative results.
  arXiv  Detail & Related papers  (2026-09-01T07:42:39Z)
• Auditing Generalization in AI-Generated Video Detection: A Six-Control Protocol and the VidAudit Toolkit [8.665845754660458]
  AI-generated video detection GenVidBench and AIGVDBench are the de facto leaderboards.<n>Most evaluation protocols leave uncontrolled confounds that can inflate reported generalizations.<n>A 20-paper survey finds none applying all six standard controls that would catch this.<n>We combine them into an audited protocol and apply it to six representative feature sources.
  arXiv  Detail & Related papers  (2026-06-30T00:29:53Z)
• Context-Aware Distillation and Ablation for Text2DSL [78.29352305480285]
  We replace prompt-only synthetic generation with context-aware distillation.<n>A teacher large language model operates under an explicitly defined structured context.<n>This scales the verified PolkitBench corpus from 4,204 to 10,073 pairs.
  arXiv  Detail & Related papers  (2026-06-21T16:27:24Z)
• Intelligence Is Not the Bottleneck: Validating an LLM First-Pass Manuscript Score Against Peer-Review Outcomes [0.0]
  Large language model (LLM) systems are increasingly proposed to assist peer review.<n>Most evaluations judge the prose of machine-generated review text, not the validity of the numeric score a system assigns.<n>We validate AIPR, which reads a submitted manuscript and emits five 0-100 quality dimensions and a weighted overall score.
  arXiv  Detail & Related papers  (2026-06-14T16:13:15Z)
• Beyond Pattern Matching: Seven Cross-Domain Techniques for Prompt Injection Detection [0.0]
  This work proposes seven detection techniques that each port a specific mechanism from a discipline outside large-language--model security.<n>Three of the seven techniques are implemented in the prompt-shield v0.4.1 release (Apache 2.0) and evaluated in a four-configuration ablation across six datasets.
  arXiv  Detail & Related papers  (2026-04-20T13:27:05Z)
• SWE-PRBench: Benchmarking AI Code Review Quality Against Pull Request Feedback [2.626039639665796]
  SWE-PRBench is a benchmark of 350 pull requests with human-annotated ground truth for evaluating AI code review quality.<n>Eight frontier models detect only 15-31% of human-flagged issues on the diff-only configuration.
  arXiv  Detail & Related papers  (2026-03-27T07:24:26Z)
• AHELM: A Holistic Evaluation of Audio-Language Models [78.20477815156484]
  multimodal audio-language models (ALMs) take interleaved audio and text as input and output text.<n>AHELM is a benchmark that aggregates various datasets -- including 2 new synthetic audio-text datasets called PARADE and CoRe-Bench.<n>We also standardize the prompts, inference parameters, and evaluation metrics to ensure equitable comparisons across models.
  arXiv  Detail & Related papers  (2025-08-29T07:40:39Z)
• Reproducible Machine Learning-based Voice Pathology Detection: Introducing the Pitch Difference Feature [1.7779568951268254]
  We introduce a novel methodology for voice pathology detection using the publicly available Saarbr"ucken Voice Database.<n>We evaluate six machine learning (ML) algorithms -- support vector machine, k-nearest neighbors, naive Bayes, decision tree, random forest, and AdaBoost.<n>Our approach 85.61%, 84.69% and 85.22% unweighted average recall (UAR) for females, males and combined results respectively.
  arXiv  Detail & Related papers  (2024-10-14T14:17:52Z)
• Exploring SSL Discrete Speech Features for Zipformer-based Contextual ASR [74.38242498079627]
  Self-supervised learning (SSL) based discrete speech representations are highly compact and domain adaptable.<n>In this paper, SSL discrete speech features extracted from WavLM models are used as additional cross-utterance acoustic context features in Zipformer-Transducer ASR systems.
  arXiv  Detail & Related papers  (2024-09-13T13:01:09Z)
• Detecting Handwritten Mathematical Terms with Sensor Based Data [71.84852429039881]
  We propose a solution to the UbiComp 2021 Challenge by Stabilo in which handwritten mathematical terms are supposed to be automatically classified. The input data set contains data of different writers, with label strings constructed from a total of 15 different possible characters.
  arXiv  Detail & Related papers  (2021-09-12T19:33:34Z)
• PATE-AAE: Incorporating Adversarial Autoencoder into Private Aggregation of Teacher Ensembles for Spoken Command Classification [53.764020085834645]
  We propose an adversarial autoencoder (AAE) to replace generative adversarial network (GAN) in the private aggregation of teacher ensembles (PATE) AAE architecture allows us to obtain good synthetic speechnative upon a discriminative training of latent vectors. PATE-AAE improves the average classification accuracy by +$2.11%$ and +$6.60%$, respectively, when compared with alternative privacy-preserving solutions.
  arXiv  Detail & Related papers  (2021-04-02T23:10:57Z)
• Evaluation Toolkit For Robustness Testing Of Automatic Essay Scoring Systems [64.4896118325552]
  We evaluate the current state-of-the-art AES models using a model adversarial evaluation scheme and associated metrics. We find that AES models are highly overstable. Even heavy modifications(as much as 25%) with content unrelated to the topic of the questions do not decrease the score produced by the models.
  arXiv  Detail & Related papers  (2020-07-14T03:49:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.