Evaluation Protocols and Cross-Subject Generalization in EEG Emotion Recognition

• URL: http://arxiv.org/abs/2607.27655v1
• Date: Thu, 30 Jul 2026 04:13:43 GMT
• Title: Evaluation Protocols and Cross-Subject Generalization in EEG Emotion Recognition
• Abstract summary: We use a dynamical graph convolutional neural network pathway on SEED and SEED-IV as an illustrative case.<n>Subject-dependent, subject-disjoint, and cross-session results should therefore be reported as answers to different questions.
• Score: 2.5603375915051747
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reported accuracy in electroencephalography (EEG) emotion recognition depends on the complete evaluation procedure, not only the classifier. We separate the target quantity, development procedure, and reporting rule, then use one archived dynamical graph convolutional neural network (DGCNN) pathway on SEED and SEED-IV as an illustrative case. In a protocol-matched subject-dependent check, the SEED result was within 1.47 percentage points of the public reference value; the 3.40-point SEED-IV difference remained unresolved. Across 30 matched SEED subject-session trajectories, checkpoint selection based on repeated test-set evaluation increased mean window accuracy from 0.7855 at epoch 80 to 0.8892. Under five-fold subject-disjoint evaluation, validation-selected checkpoints achieved training-participant trial accuracies of 0.9990 on SEED and 0.9920 on SEED-IV. Accuracy for entirely held-out participants was 0.5348 (95% conditional subject-level bias-corrected and accelerated [BCa] interval [0.4667, 0.5985]) on SEED. The SEED-IV estimate was 0.3954 ([0.3343, 0.4648]) and is reported only as secondary sensitivity evidence because its protocol-matched compatibility check remained unresolved. The observed train-to-held-out-subject gaps are inconsistent with simple optimization underfitting, but they do not isolate subject identity from implementation, preprocessing, representation, or distributional factors. Supporting analyses further showed that participant rankings depended on representation and time scale, while a development-selected tail-risk ensemble did not establish a positive gain in a separate final evaluation. Subject-dependent, subject-disjoint, and cross-session results should therefore be reported as answers to different questions.

Related papers

• The Model Proposes, the Code Disposes: A Pre-Registered Ablation of a Verifier-and-Acceptance Stage in an LLM-Orchestrated Offensive-Security Agent [0.0]
  We evaluate whether a verifier-and-acceptance stage - a model verifier whose verdicts are enforced by deterministic code - changes what an LLM-driven offensive-security agent reports.<n>We report a 15-run exploratory pilot, a pre-registered 20-run confirmatory ablation, and a pre-registered 2 x 2 factorial study with 40 runs across two deliberately vulnerable lab targets.<n>The results support a narrow conclusion: the verifier changes what the system ships, while deterministic code supplies enforcement and auditability.
  arXiv  Detail & Related papers  (2026-09-14T17:13:05Z)
• Vision-Language Models for Criterion-Level Grading of Handwritten Examinations in Outcome-Based Education [0.3686530147760243]
  This study evaluates vision-language models for handwritten outcome-based assessment.<n>We compare 20 configurations spanning Qwen2.5-VL, InternVL3, Pixtral, a Donut baseline, and a cascade ensemble.
  arXiv  Detail & Related papers  (2026-09-13T04:55:55Z)
• Counterfactual Fairness Audits of Multi-Step Clinical LLM Agents Require a Measured Per-Action Instability Floor [3.0704370207786265]
  We show that a counterfactual fairness estimate reported without a per-action floor beside it cannot be read as evidence of disparity.<n>The measurements were taken with FairMedAgent, an evaluation harness for disparity in the actions of clinical adjudication agents.
  arXiv  Detail & Related papers  (2026-09-02T23:38:17Z)
• Beyond Local Accuracy: A Protocol-Level Identifiability Audit for Controlled LLM Reasoning Evaluation [6.047519836191728]
  We formalize a protocol-level identifiability audit over a finite behavioral policy class.<n>The audit requires zero model calls and resolves our diagnostic case.<n>This case shows how evaluation-design validity can be checked structurally before model inference.
  arXiv  Detail & Related papers  (2026-08-13T14:49:47Z)
• Automated binary classification of hazelnut X-ray images: A deep-learning benchmark for quality assessment [33.72751145910978]
  Non-destructive X-ray imaging can reveal internal hazelnut defects that are difficult to detect by external inspection alone.<n>We present a benchmark for binary hazelnut quality classification based on 799 segmented single- Kernel X-ray images.
  arXiv  Detail & Related papers  (2026-08-12T07:55:22Z)
• From Checker to Forecaster: Code-Owned Evaluation of Model-Generated Strategic Routes Under Delayed Ground Truth [0.0]
  RouteCast instantiates the regime for model-generated typed strategic routes.<n>It produces a provisional forecast-ranking; later outcomes evaluate the forecast.<n>It does not establish prospective calibration, causal decision improvement, route-decomposition advantage, or cross-domain validity.
  arXiv  Detail & Related papers  (2026-07-13T00:36:48Z)
• Training-Free Model Selection and Domain-Aware Score Calibration for First-Shot Anomalous Sound Detection [0.12691047660244334]
  First-shot anomalous sound detection in DCASE Challenge Task 2 must flag anomalies of unseen machine types with a single threshold.<n>Two organizer-reported problems remain open: source- and target-domain AUC are negatively correlated across systems, and development-set performance does not predict evaluation-set performance.<n>We address both with a training-free post-hoc layer over frozen audio embeddings.
  arXiv  Detail & Related papers  (2026-07-05T22:22:31Z)
• Distribution-Free Uncertainty Quantification for Continuous AI Agent Evaluation [1.2299000423193074]
  We adapt split conformal prediction and adaptive conformal inference to continuous AI agent evaluation.<n>We show that per-agent conditional coverage is well-concentrated around the nominal level.<n>A circularity-controlled validation confirms the framework captures signal beyond benchmarks.
  arXiv  Detail & Related papers  (2026-05-19T12:47:21Z)
• A Nonlinear Complexity Index for Wearable PPG Cardiovascular Stability: Multiscale Validation, Systematic Evaluation Correction, and Bayesian Parameter Optimization [1.1344265020822928]
  We introduce a Stability-Constrained Cardiovascular Stability Index (SCSI) grounded in Cardiac Stability and validate it across 176,742 segments from four heterogeneous temporal scales.<n>We identify three evaluation artifacts that inflate component from true baseline to 0.5732: segment-level cross-validation leakage, test-set leakage, and pooled overweighting that conceals per-patient failure.
  arXiv  Detail & Related papers  (2026-05-11T13:55:30Z)
• From Black Box to Glass Box: Cross-Model ASR Disagreement to Prioto Review in Ambient AI Scribe Documentation [43.148402136307716]
  Cross-model disagreement among heterogeneous ASR systems can act as a reference-free uncertainty signal.<n>We transcribed 50 publicly available medical audio clips with eight ASR systems spanning commercial APIs and open-source engines.<n>Low-agreement regions were enriched for content disagreements, with the content fraction increasing from 53.9% to 73.9% across quintiles of high-risk mass.
  arXiv  Detail & Related papers  (2026-03-02T13:02:13Z)
• Development of a fully deep learning model to improve the reproducibility of sector classification systems for predicting unerupted maxillary canine likelihood of impaction [0.11726720776908518]
  The aim of the present study was to develop a fully deep learning model to reduce the intra- and inter-operator of sector classification systems.<n>DenseNet121 proved to be the best-performing model in allocating impacted in the three different classes, with an overall accuracy of 76.8%.
  arXiv  Detail & Related papers  (2025-11-24T14:45:46Z)
• SummExecEdit: A Factual Consistency Benchmark in Summarization with Executable Edits [31.98028879922584]
  We introduce SummExecEdit, a novel pipeline and benchmark to assess models on their ability to both detect factual errors and provide accurate explanations.<n>The top-performing model, Claude3-Opus, achieves a joint detection and explanation score of only 0.49 in our benchmark.<n>We identify four primary types of explanation errors, with 45.4% of them involving a focus on completely unrelated parts of the summary.
  arXiv  Detail & Related papers  (2024-12-17T23:26:44Z)
• CardioSyntax: end-to-end SYNTAX score prediction -- dataset, benchmark and method [36.440495538328754]
  The SYNTAX score has become a widely used measure of coronary disease severity.<n>This paper introduces a new medical regression and classification problem - automatically estimating SYNTAX score from coronary angiography.<n>We present a comprehensive CardioSYNTAX dataset of 3,018 patients for the SYNTAX score estimation and coronary dominance classification.
  arXiv  Detail & Related papers  (2024-07-29T11:16:59Z)
• Systematic Clinical Evaluation of A Deep Learning Method for Medical Image Segmentation: Radiosurgery Application [48.89674088331313]
  We systematically evaluate a Deep Learning (DL) method in a 3D medical image segmentation task. Our method is integrated into the radiosurgery treatment process and directly impacts the clinical workflow.
  arXiv  Detail & Related papers  (2021-08-21T16:15:40Z)
• Correct block-design experiments mitigate temporal correlation bias in EEG classification [68.85562949901077]
  We show that the main claim in [1] is drastically overstated and their other analyses are seriously flawed by wrong methodological choices. We investigate the influence of EEG temporal correlation on classification accuracy by testing the same models in two additional experimental settings.
  arXiv  Detail & Related papers  (2020-11-25T22:25:21Z)
• Multilabel 12-Lead Electrocardiogram Classification Using Gradient Boosting Tree Ensemble [64.29529357862955]
  We build an algorithm using gradient boosted tree ensembles fitted on morphology and signal processing features to classify ECG diagnosis. For each lead, we derive features from heart rate variability, PQRST template shape, and the full signal waveform. We join the features of all 12 leads to fit an ensemble of gradient boosting decision trees to predict probabilities of ECG instances belonging to each class.
  arXiv  Detail & Related papers  (2020-10-21T18:11:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.