Automating Learner Assessment: Benchmarking Machine Learning and Deep Learning Models for EEG-Based Familiarity Prediction

• URL: http://arxiv.org/abs/2608.16541v2
• Date: Thu, 20 Aug 2026 13:12:52 GMT
• Title: Automating Learner Assessment: Benchmarking Machine Learning and Deep Learning Models for EEG-Based Familiarity Prediction
• Abstract summary: This study benchmarks fifteen machine learning (ML) and deep learning (DL) models for EEG-based familiarity prediction.<n>We show that while standard stratified cross-validation yields artificially high classification performance (up to 0.9853 F1-score using CNN) due to temporal leakage across neighboring epochs, a rigorous trial-independent validation (Group K-Fold) drops the peak performance to 0.6038 F1-score (using CNN)<n>This highlights the critical necessity of trial-independent evaluation to avoid overestimating model generalizability.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Objective assessment of learning remains a fundamental challenge in education. Electroencephalography (EEG) provides a direct, non-invasive window into the neural correlates of knowledge acquisition, including cognitive familiarity. This study benchmarks fifteen machine learning (ML) and deep learning (DL) models for EEG-based familiarity prediction across two cognitive domains: faces (factual knowledge) and mathematical equations (conceptual knowledge). Using continuous EEG data from 23 participants, we extract spectral features (Power Spectral Density) across six frequency bands. We show that while standard stratified cross-validation yields artificially high classification performance (up to 0.9853 F1-score using CNN) due to temporal leakage across neighboring epochs, a rigorous trial-independent validation (Group K-Fold) drops the peak performance to 0.6038 F1-score (using CNN), which is still statistically significant above the 25% chance level. This highlights the critical necessity of trial-independent evaluation to avoid overestimating model generalizability. Furthermore, feature importance and SHAP analysis reveal that temporal and frontal Gamma and Beta oscillations are the most critical biomarkers for familiarity. This work establishes a realistic benchmark for EEG-based cognitive monitoring in educational technologies.

Related papers

• Taming foundation model with invariance-oriented pre-training for broad-spectrum EEG analysis across signal-level, brain-state, and brain-health tasks [46.533103231728724]
  INCEPT is an invariance-oriented EEG foundation model trained on over 11,000 hours of unlabelled clinical EEG.<n>We evaluate INCEPT on a broad-spectrum benchmark of ten datasets spanning three levels of post-acquisition EEG analysis.
  arXiv  Detail & Related papers  (2026-08-25T14:20:02Z)
• Automated Detection of Abnormalities in Zebrafish Development [34.13586231002437]
  We introduce a large-scale dataset of high-resolution microscopic image sequences of zebrafish embryonic development under both conditions and exposure to compounds.<n>This dataset supports two benchmarking tasks: (1) Zebra fertility classification, assessing zebrafish egg viability (130,368 images), and (2) toxicity assessment, detecting malformations induced by toxic exposure over time (55,296 images)<n>Alongside the dataset, we present the first transformer-based baseline model that integrates zebrafish egg features to predict developmental abnormalities at the early stages.
  arXiv  Detail & Related papers  (2026-05-11T12:33:04Z)
• Demographic-Aware Self-Supervised Anomaly Detection Pretraining for Equitable Rare Cardiac Diagnosis [45.00216485193071]
  Rare cardiac anomalies are difficult to detect from electrocardiograms due to their long-tailed distribution.<n>We developed an AI-assisted two-stage ECG framework integrating self-supervised anomaly detection and demographic-aware representation learning.<n>Our method achieves an AUROC of 94.7% for rare anomalies and reduces the common-rare performance gap by 73%.
  arXiv  Detail & Related papers  (2026-03-20T07:04:00Z)
• Detecting Structural Heart Disease from Electrocardiograms via a Generalized Additive Model of Interpretable Foundation-Model Predictors [8.817617912039616]
  Structural heart disease (SHD) is a prevalent condition with many undiagnosed cases.<n>Recent studies show that artificial intelligence (AI)-based analysis of electrocardiograms (ECGs) can detect SHD.<n>Existing methods are fully black-box models, limiting interpretability and clinical adoption.
  arXiv  Detail & Related papers  (2026-03-03T05:39:32Z)
• Attention-Based Deep Learning for Early Parkinson's Disease Detection with Tabular Biomedical Data [0.0]
  Early and accurate detection of Parkinson's disease (PD) remains a critical challenge in medical diagnostics.<n>Traditional machine learning (ML) models, though widely applied to PD detection, often rely on extensive feature engineering and struggle to capture complex feature interactions.<n>We present a comparative evaluation of four classification models: Multi-Layer Perceptron (MLP), Gradient Boosting, TabNet, and SAINT.
  arXiv  Detail & Related papers  (2026-02-08T12:03:02Z)
• The role of data partitioning on the performance of EEG-based deep learning models in supervised cross-subject analysis: a preliminary study [37.69303106863453]
  Deep learning is advancing the analysis of electroencephalography (EEG) data by effectively discovering highly nonlinear patterns.<n>No comprehensive guidelines for proper data partitioning and cross-validation exist in the domain.<n>This paper thoroughly investigates the role of data partitioning and cross-validation in evaluating EEG deep learning models.
  arXiv  Detail & Related papers  (2025-05-19T12:05:28Z)
• BioSerenity-E1: a self-supervised EEG model for medical applications [0.0]
  BioSerenity-E1 is a family of self-supervised foundation models for clinical EEG applications.<n>It combines spectral tokenization with masked prediction to achieve state-of-the-art performance across relevant diagnostic tasks.
  arXiv  Detail & Related papers  (2025-03-13T13:42:46Z)
• EVolutionary Independent DEtermiNistiC Explanation [5.127310126394387]
  This paper introduces the Evolutionary Independent Deterministic Explanation (EVIDENCE) theory.<n>EVIDENCE offers a deterministic, model-independent method for extracting significant signals from black-box models.<n> Practical applications of EVIDENCE include improving diagnostic accuracy in healthcare and enhancing audio signal analysis.
  arXiv  Detail & Related papers  (2025-01-20T12:05:14Z)
• Uncertainty-inspired Open Set Learning for Retinal Anomaly Identification [71.06194656633447]
  We establish an uncertainty-inspired open-set (UIOS) model, which was trained with fundus images of 9 retinal conditions. Our UIOS model with thresholding strategy achieved an F1 score of 99.55%, 97.01% and 91.91% for the internal testing set. UIOS correctly predicted high uncertainty scores, which would prompt the need for a manual check in the datasets of non-target categories retinal diseases, low-quality fundus images, and non-fundus images.
  arXiv  Detail & Related papers  (2023-04-08T10:47:41Z)
• Explainable Censored Learning: Finding Critical Features with Long Term Prognostic Values for Survival Prediction [28.943631598055926]
  We introduce a novel, easily deployable approach, called EXplainable CEnsored Learning (EXCEL), to iteratively exploit critical variables. We show that EXCEL can effectively identify critical features and achieve performance on par with or better than the original models.
  arXiv  Detail & Related papers  (2022-09-30T12:56:29Z)
• Global ECG Classification by Self-Operational Neural Networks with Feature Injection [25.15075119957447]
  We propose a novel approach for inter-patient ECG classification using a compact 1D Self-Organized Operational Neural Networks (Self-ONNs) We used 1D Self-ONN layers to automatically learn morphological representations from ECG data, enabling us to capture the shape of the ECG waveform around the R peaks. Using the MIT-BIH arrhythmia benchmark database, the proposed method achieves the highest classification performance ever achieved.
  arXiv  Detail & Related papers  (2022-04-07T22:49:18Z)
• Evaluation of Interpretability for Deep Learning algorithms in EEG Emotion Recognition: A case study in Autism [4.752074022068791]
  Current models on Explainable Artificial Intelligence (XAI) have shown an evident and quantified lack of reliability for measuring feature-relevance. This study is the first to consolidate a more transparent feature-relevance calculation for a successful EEG-based facial emotion recognition.
  arXiv  Detail & Related papers  (2021-11-25T18:28:29Z)
• Real-Time Patient-Specific ECG Classification by 1D Self-Operational Neural Networks [24.226952040270564]
  We propose 1D Self-organized Operational Neural Networks (1D Self-ONNs) for ECG classification. 1D Self-ONNs have the utmost advantage and superiority over conventional ONNs where the prior operator search within the operator set library is entirely avoided. Our results over the MIT-BIH arrhythmia benchmark database demonstrate that 1D Self-ONNs can surpass 1D CNNs with a significant margin.
  arXiv  Detail & Related papers  (2021-09-30T19:37:36Z)
• Uncovering the structure of clinical EEG signals with self-supervised learning [64.4754948595556]
  Supervised learning paradigms are often limited by the amount of labeled data that is available. This phenomenon is particularly problematic in clinically-relevant data, such as electroencephalography (EEG) By extracting information from unlabeled data, it might be possible to reach competitive performance with deep neural networks.
  arXiv  Detail & Related papers  (2020-07-31T14:34:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.