A Controlled Synthetic Benchmark for Educational Aspect-Based Sentiment Analysis

• URL: http://arxiv.org/abs/2605.25502v1
• Date: Mon, 25 May 2026 07:05:21 GMT
• Title: A Controlled Synthetic Benchmark for Educational Aspect-Based Sentiment Analysis
• Abstract summary: This study introduces a controlled benchmark for educational ABSA built from 10,000 synthetic course reviews.<n>The strongest untuned model, BERT, reaches a held-out detection micro-F1 of 0.2760.<n>A conservative external evaluation on 2,829 mapped student-feedback reviews from Herath et al. yields a micro-F1 of 0.4593 for BERT on a 9-aspect overlap.
• Score: 47.027290803102666
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Educational aspect-based sentiment analysis (ABSA) can support course improvement, but public aspect-labeled student feedback remains scarce because educational reviews are private, institution-specific, and expensive to annotate. This study introduces a controlled synthetic benchmark for educational ABSA built from 10,000 synthetic course reviews with explicit train-validation-test splits and a 20-aspect pedagogical schema spanning instructional quality, assessment and course management, learning demand, learning environment, and engagement. The corpus is generated with sampled target labels, sampled nuance attributes, and a realism-tuned prompt refined through a three-cycle judge-editor procedure. On the resulting benchmark, local baselines with TF-IDF, two-step transformers, and joint encoders show that the task is nontrivial; the strongest untuned model, BERT, reaches a held-out detection micro-F1 of 0.2760, while a modest lower-rate BERT schedule improves this to 0.2930. Full-test GPT-based inference with gpt-5.2 reaches 0.2519 micro-F1 in zero-shot mode and 0.2501 with retrieval-based few-shot prompting, placing batch inference above the classical baseline and close to the compact joint encoders. A conservative external evaluation on 2,829 mapped student-feedback reviews from Herath et al. yields a micro-F1 of 0.4593 for BERT on a 9-aspect overlap, indicating partial synthetic-to-real transfer. Realism and faithfulness analyses are reported as generator diagnostics that clarify how the benchmark was stabilized and where label noise remains. The study therefore contributes a synthetic educational ABSA corpus, a documented generation procedure, and a reproducible benchmark setting for a domain in which public labeled data remain difficult to obtain.

Related papers

• Empirical Evaluation of Membership Inference Attacks on NLP Text Classifiers: A Baseline Study on SST-2 [0.0]
  Membership inference attacks (MIAs) try to determine whether a specific record was used to train a model.<n>This paper presents a benchmark of membership inference vulnerability for text classification on the GLUE SST-2 sentiment dataset.
  arXiv  Detail & Related papers  (2026-09-10T00:53:20Z)
• Rethinking the Evaluation and Optimization of LLM-Based Social Simulation [56.18577997572273]
  We use the subjectivity coefficient to analyze how accuracy-based evaluation and hard-label training fail as subjectivity grows.<n>We construct SUBJSIM, a benchmark of 19,300 contexts covering 193 annotators and 100 subjective questions.<n>Results on SUBJSIM demonstrate the advantages of our method.
  arXiv  Detail & Related papers  (2026-08-20T06:26:03Z)
• BioSentinel at EXIST 2026: Soft-Label Optimization with XLM-RoBERTa for Sexism Intent Classification in Memes [0.8749675983608171]
  This paper describes the BioSentinel team's participation in EXIST 2026 Task 2.2: Source Intention in Memes.<n>The task requires classifying the communicative intent behind memes as direct, judgemental, or no (non-sexist)<n>We present a text-centric approach built on xlm-roberta-base (270M parameters) trained with a composite loss function combining KL divergence on soft annotator distributions.
  arXiv  Detail & Related papers  (2026-07-27T08:18:05Z)
• Verification and Identification in ECG biometric on large-scale [0.0]
  This work studies electrocardiogram (ECG) biometrics at large scale.<n>We address a critical gap in the literature: the scarcity of large-scale evaluations with operational metrics and protocols.
  arXiv  Detail & Related papers  (2026-02-02T20:30:35Z)
• Automatic Essay Scoring and Feedback Generation in Basque Language Learning [4.218073067465283]
  This paper introduces the first publicly available dataset for Automatic Essay Scoring (AES) and feedback generation in Basque, targeting the CEFR C1 proficiency level.<n>The dataset comprises 3,200 essays from HABE, each annotated by expert evaluators with criterion specific scores covering correctness, richness, coherence, cohesion, and task alignment enriched with detailed feedback and error examples.<n>We fine-tune open-source models, including RoBERTa-EusCrawl and Latxa 8B/70B, for both scoring and explanation generation.
  arXiv  Detail & Related papers  (2025-12-09T15:28:35Z)
• Evaluating the Ability of Large Language Models to Identify Adherence to CONSORT Reporting Guidelines in Randomized Controlled Trials: A Methodological Evaluation Study [7.142913983218931]
  This study aimed to evaluate the accuracy and reliability of contemporary LLMs in identifying adherence of published RCTs.<n>We constructed a golden standard dataset of 150 published RCTs spanning diverse medical specialties.<n>Top-performing models, Gemini-2.5-Flash and DeepSeek-R1, achieved nearly identical macro F1 scores of 0.634 and Cohen's Kappa coefficients of 0.280 and 0.282, respectively.
  arXiv  Detail & Related papers  (2025-11-17T08:05:15Z)
• Why Registration Quality Matters: Enhancing sCT Synthesis with IMPACT-Based Registration [1.2560645967579729]
  Our model is a 2.5D U-Net++ with a ResNet-34 encoder, trained jointly across anatomical regions and fine-tuned per region.<n>On the local test sets, IMPACT-based registration achieved more accurate and anatomically consistent alignments than mutual-information-based registration.
  arXiv  Detail & Related papers  (2025-10-24T11:40:21Z)
• LLMEval-3: A Large-Scale Longitudinal Study on Robust and Fair Evaluation of Large Language Models [51.55869466207234]
  Existing evaluation of Large Language Models (LLMs) on static benchmarks is vulnerable to data contamination and leaderboard overfitting.<n>We introduce LLMEval-3, a framework for dynamic evaluation of LLMs.<n>LLEval-3 is built on a proprietary bank of 220k graduate-level questions, from which it dynamically samples unseen test sets for each evaluation run.
  arXiv  Detail & Related papers  (2025-08-07T14:46:30Z)
• Quantifying Holistic Review: A Multi-Modal Approach to College Admissions Prediction [1.9594639581421427]
  Comprehensive Applicant Profile Score (CAPS) is a novel framework designed to quantitatively model and interpret holistic college admissions evaluations.<n>CAPS decomposes applicant profiles into three interpretable components: academic performance, essay quality, and extracurricular engagement.
  arXiv  Detail & Related papers  (2025-07-12T16:58:03Z)
• T2I-Eval-R1: Reinforcement Learning-Driven Reasoning for Interpretable Text-to-Image Evaluation [60.620408007636016]
  We propose T2I-Eval-R1, a novel reinforcement learning framework that trains open-source MLLMs using only coarse-grained quality scores.<n>Our approach integrates Group Relative Policy Optimization into the instruction-tuning process, enabling models to generate both scalar scores and interpretable reasoning chains.
  arXiv  Detail & Related papers  (2025-05-23T13:44:59Z)
• Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss [72.62029620566925]
  Recent works in self-supervised learning have advanced the state-of-the-art by relying on the contrastive learning paradigm. Our work analyzes contrastive learning without assuming conditional independence of positive pairs. We propose a loss that performs spectral decomposition on the population augmentation graph and can be succinctly written as a contrastive learning objective.
  arXiv  Detail & Related papers  (2021-06-08T07:41:02Z)
• Exploiting Sample Uncertainty for Domain Adaptive Person Re-Identification [137.9939571408506]
  We estimate and exploit the credibility of the assigned pseudo-label of each sample to alleviate the influence of noisy labels. Our uncertainty-guided optimization brings significant improvement and achieves the state-of-the-art performance on benchmark datasets.
  arXiv  Detail & Related papers  (2020-12-16T04:09:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.