Average Rankings Mask Per-Subject Optimality: A Friedman-Nemenyi Benchmark of EEG Motor-Imagery BCI Decoders

• URL: http://arxiv.org/abs/2606.24394v1
• Date: Tue, 23 Jun 2026 10:26:27 GMT
• Title: Average Rankings Mask Per-Subject Optimality: A Friedman-Nemenyi Benchmark of EEG Motor-Imagery BCI Decoders
• Abstract summary: We evaluate 1,056 decoding configurations and 340,000 subject-level models.<n>Every model is fit and tested within a single session of a single participant.<n>Even in the easiest regime, no single pipeline dominates.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Electroencephalography (EEG) is the dominant non-invasive modality for brain-computer interfaces (BCIs), yet reliable decoding of motor imagery is hampered by inter- and intra-individual variability. A recurring claim is that one decoding pipeline, most often a spatial or Riemannian method, is broadly preferable. We test the weakest version of that claim under the most favourable conditions. Using the Mother of All BCI Benchmarks (MOABB) framework, we evaluated 1,056 decoding configurations (feature extractor x scaler x classifier), >340,000 subject-level model fits, across three public left-versus-right motor-imagery datasets (PhysionetMI, 109 participants; Cho2017, 52; Zhou2016, 4) and two frequency bands (8-15 Hz, 8-30 Hz). Every model is fit and tested within a single session of a single participant, the easiest regime, giving every pipeline its best chance. We apply the statistics standard for multi-classifier comparison: Friedman omnibus tests, Nemenyi critical-difference analysis and Wilcoxon signed-rank tests with effect sizes. Covariance tangent-space projection (cov-tgsp) and Common Spatial Patterns (CSP) are the strongest families, but their ordering is dataset-dependent and, on the largest and most heterogeneous cohort (PhysionetMI), statistically indistinguishable (Nemenyi p = 0.27; Kendall's W = 0.11). At the individual level the single best pipeline is optimal for only 35% of PhysionetMI participants, and nonlinear descriptors are best for roughly one third; matching pipeline to participant adds about seven accuracy points over the best fixed choice. The ranking is not an artefact of dimensionality, and classifier and scaler choices are secondary to the feature representation. Even in the easiest regime, no single pipeline dominates: a lower bound on the personalization problem and a quantitative case for participant-aware model selection rather than a universal decoder.

Related papers

• ER-KANs: Efficient and Robust Kolmogorov-Arnold Networks for Data-Scarce Scientific Machine Learning [0.0]
  We show that ChebyKAN's test MSE increases by a factor of 10.6x when training data is corrupted with sigma=0.1 noise.<n> ER-KAN combines three design choices targeting the noisy, data-scarce setting.
  arXiv  Detail & Related papers  (2026-08-14T16:15:37Z)
• Multimodal Ambivalence and Hesitancy Recognition via Cross-Attention and Gated Fusion [1.6950215926321557]
  We present a framework for Ambivalence/Hesitancy (A/H) recognition in video, developed for the ABAW11 challenge at ECCV 2026.<n>The proposed approach fuses textual, acoustic, and visual modalities extracted from the BAH dataset.<n>This model achieves a Macro F1 of textbf0.7394 on the validation set, a relative improvement of 11.0% over the best unimodal baseline.
  arXiv  Detail & Related papers  (2026-07-17T09:23:36Z)
• VendorBench-100: A Unified Cross-Paradigm Benchmark for Deepfake Image Detection [0.04736448323490553]
  VendorBench-100 is a cross-paradigm benchmark for deepfake image detection.<n>It evaluates 36 representative models using a single adversarial 100-image corpus, a unified output schema, and a common evaluation framework.<n>Our evaluation shows that commercial APIs achieve the strongest median performance, followed by vision LLMs and open-source detectors.
  arXiv  Detail & Related papers  (2026-07-07T13:22:00Z)
• SciOrch: Learning to Orchestrate Expert LLMs for Solving Frontier Multimodal Scientific Reasoning Tasks [65.14594927081983]
  We present SciOrch, a framework that trains a lightweight 8B model to orchestrate frontier LLMs for scientific reasoning.<n>On a 240-question test set spanning SGI-Reasoning and Scientists' First exam, SciOrch reaches 56.66% average accuracy.<n>It also attains the best accuracy on both SGI and SFE with less than half the API cost of typical multi-agent methods.
  arXiv  Detail & Related papers  (2026-06-14T15:45:34Z)
• Off-the-Shelf LLMs as Process Scorers: Training-Free Alternative to PRMs for Mathematical Reasoning [51.88950852117154]
  Chunk-Level Guided Generation is a training-free alternative that uses an off-the-shelf large language model as a process scorer.<n>We show that scoring variable-length reasoning steps with large-model likelihoods is unreliable due to a systematic length bias.<n>Chunk-Level Guided Generation produces substantially shorter reasoning traces than PRM guided search.
  arXiv  Detail & Related papers  (2026-06-01T04:43:36Z)
• Submodular Benchmark Selection [51.56484100374058]
  Entropy and mutual information between selected and remaining benchmarks arise as natural objectives.<n>Experiments on three matrices from ten public leaderboards show that mutual information selection outperforms entropy for imputation at small subsets.
  arXiv  Detail & Related papers  (2026-05-04T04:17:02Z)
• Rethinking Generalized BCIs: Benchmarking 340,000+ Unique Algorithmic Configurations for EEG Mental Command Decoding [0.0]
  We present a benchmark evaluating over 340,000+ unique combinations of spatial and nonlinear EEG classification.<n>Our findings highlight that no universal 'one-size-fits-all' method can optimally decode EEG motor imagery patterns across all users or datasets.
  arXiv  Detail & Related papers  (2025-12-02T17:56:46Z)
• DISCO: Diversifying Sample Condensation for Efficient Model Evaluation [59.01400190971061]
  Costly evaluation reduces inclusivity, slows the cycle of innovation, and worsens environmental impact.<n>We argue that promoting diversity among samples is not essential; what matters is to select samples thatmaximise diversity in model responses.<n>Our method, $textbfDiversifying Sample Condensation (DISCO)$, selects the top-k samples with the greatest model disagreements.
  arXiv  Detail & Related papers  (2025-10-09T08:53:59Z)
• MedSegMamba: 3D CNN-Mamba Hybrid Architecture for Brain Segmentation [15.514511820130474]
  We develop a 3D patch-based hybrid CNN-Mamba model for subcortical brain segmentation. Our model's performance was validated against several benchmarks.
  arXiv  Detail & Related papers  (2024-09-12T02:19:19Z)
• WaferSegClassNet -- A Light-weight Network for Classification and Segmentation of Semiconductor Wafer Defects [3.1806743741013648]
  We present WaferSegClassNet (WSCN), a novel network based on encoder-decoder architecture. WSCN performs simultaneous classification and segmentation of both single and mixed-type wafer defects. We are the first to show segmentation results on the MixedWM38 dataset.
  arXiv  Detail & Related papers  (2022-07-03T05:46:19Z)
• Least Squares Regression with Markovian Data: Fundamental Limits and Algorithms [69.45237691598774]
  We study the problem of least squares linear regression where the data-points are dependent and are sampled from a Markov chain. We establish sharp information theoretic minimax lower bounds for this problem in terms of $tau_mathsfmix$. We propose an algorithm based on experience replay--a popular reinforcement learning technique--that achieves a significantly better error rate.
  arXiv  Detail & Related papers  (2020-06-16T04:26:50Z)
• AvgOut: A Simple Output-Probability Measure to Eliminate Dull Responses [97.50616524350123]
  We build dialogue models that are dynamically aware of what utterances or tokens are dull without any feature-engineering. The first model, MinAvgOut, directly maximizes the diversity score through the output distributions of each batch. The second model, Label Fine-Tuning (LFT), prepends to the source sequence a label continuously scaled by the diversity score to control the diversity level. The third model, RL, adopts Reinforcement Learning and treats the diversity score as a reward signal.
  arXiv  Detail & Related papers  (2020-01-15T18:32:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.