How Much Capacity Does EEG Denoising Need? Ultra-Compact Networks reveal Benchmark Saturation and Metric-Utility Gap

• URL: http://arxiv.org/abs/2606.08594v2
• Date: Mon, 15 Jun 2026 12:51:03 GMT
• Title: How Much Capacity Does EEG Denoising Need? Ultra-Compact Networks reveal Benchmark Saturation and Metric-Utility Gap
• Abstract summary: Deep learning EEG denoising architectures have scaled from tens of thousands to tens of millions of parameters, yet no prior study has isolated model capacity as the experimental variable.<n>We address both gaps by fixing architecture, loss, data split, and training recipe while sweeping only channel width from 1.05K to 40.26K parameters in a minimal depthwiseseparable convolutional U-Net.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep learning EEG denoising architectures have scaled from tens of thousands to tens of millions of parameters, yet no prior study has isolated model capacity as the experimental variable or tested whether reconstruction metrics predict downstream neural-signal utility. We address both gaps by fixing architecture, loss, data split, and training recipe while sweeping only channel width from 1.05K to 40.26K parameters in a minimal depthwise-separable convolutional U-Net. Models were evaluated on the EEGDenoiseNet benchmark, cross-dataset BCI transfer tests, controlled baseline retraining, and downstream motor-imagery classification with five decoder families across all nine BCI Competition IV-2a subjects. Reconstruction performance saturated by 3-6.5K parameters, with post-elbow gains of at most 0.015 correlation coefficient per log10-parameter unit. An 8.46M-parameter baseline retrained under the same pipeline matched the 40.26K compact variant on EOG--a 200x parameter gap yielding no advantage--while a Patch-Transformer control reproduced the same diminishing-return shape. Downstream evaluation exposed a classifier-dependent metric-utility gap: reconstruction-optimized denoising significantly degraded CSP+LDA classification across all nine subjects and three artifact types (best denoised accuracy 0.547 vs. 0.612 noisy baseline; Bonferroni p=0.0488), persisting on naturally recorded trials (Delta=-0.047; BH-FDR q=0.0049). End-to-end neural decoders showed variable or neutral effects. Standard EEG denoising benchmarks are saturated far below current model capacity, and reconstruction metrics do not predict BCI utility. Ultra-compact models at 33-46 KB and 1.27-2.61M FLOPs/segment are practical for edge deployment. These findings argue for capacity-controlled evaluation, harder task-aware benchmarks, and mandatory downstream validation.

Related papers

• On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability [57.03393882933515]
  We describe the architecture and ablations of Qwen3.8-Flash-Next, a sparse mixture-of-experts model with 125B parameters, 6B activated per token, and 51B parameters of n-gram embedding tables held off the accelerator.<n>On fourteen pre-training benchmarks the model leads the 397B-A17B predecessor on eight and trails it on the rest by at most 2.6 points, at 1/3 the activated parameters, 1/3 the training tokens, and roughly 1/9 the training FLOPs.
  arXiv  Detail & Related papers  (2026-08-31T06:35:07Z)
• Quantum-Inspired Hybrid Neural Networks for Neural Decoding: A Controlled Ablation Study of Learnable Quantum Sidecar Integration [5.2726003549534655]
  We study parameterized quantum circuits (PQCs) integrated as residual sidecar modules within a ResNet-50 backbone for neural population decoding.<n>We compare four model variants: baseline, quantum sidecar with frozen input projection, quantum sidecar with backbone-gradient-trained projection, and a measurement-guided variant that aligns angle encodings with circuit measurement outcomes.
  arXiv  Detail & Related papers  (2026-08-23T15:55:43Z)
• Leakage-Robust Evaluation and Data-Scale Sensitivity of Attention-Enhanced Multi-Task Learning for Joint Fault Diagnosis and Remaining Useful Life Estimation [0.0]
  Multi-task deep learning models can be affected by how sliding-window sequences are split into training and test sets.<n>We show that naive splitting can inflate classification accuracy from a genuine 20-60 percent to 99.9 percent.<n>We introduce a chunk-based, leakage-audited splitting protocol and evaluate all models using five seeds, one-way ANOVA, and Tukey HSD tests.
  arXiv  Detail & Related papers  (2026-07-17T20:33:36Z)
• Scaling Closed-Loop Feature Channel Configuration with LLMs [48.83701310501069]
  The analysis covers 2000 generated candidates from 8 complete cycles, yielding 462 verified CIFAR-100 evaluations after task and metadata filtering.<n>The best model reaches 0.3676 with 11.8M parameters, compared with an early high-performing model at 0.3144 with 166.5M parameters.<n>Non-power-of-two channel widths occur in 41.8% of verified candidates, and the strongest models share structured channelallocation patterns.
  arXiv  Detail & Related papers  (2026-07-07T22:31:53Z)
• Towards Robust Training in NNGPT AutoML Pipeline: A Loss-Optimizer Pairing Selection Study [48.83701310501069]
  This paper investigates whether a single recipe is sufficient for heterogeneous architecture pools, or whether the optimal pairing varies across structurally diverse models.<n>We conduct a systematic empirical study of all $3 times 6 = 18$ combinations of six classifiers paired with three loss functions: Cross-Entropy (CEL), Negative Log-Likelihood (NLL), and the recently introduced genetically evolved NGL loss across the base models presented in LEMUR heterogeneous architecture pool on six image classification datasets.<n>Our results confirm that no single pairing is universally optimal. Cross-Entropy with Adam or AdamW is the most robust choice across
  arXiv  Detail & Related papers  (2026-06-18T20:51:42Z)
• KPGrasp: Scalable Keypoint Flow Matching for Dexterous Grasp Generation [23.028638559051597]
  We present KPGrasp, a flow-matching framework that learns dexterous grasp priors from large-scale data.<n>Experiments on 20 diverse objects demonstrate that the pipeline can be deployed in a real-world setup.
  arXiv  Detail & Related papers  (2026-06-08T10:19:29Z)
• Normalized Architectures are Natively 4-Bit [49.13186675123547]
  nGPT, an architecture that constrains weights and hidden representations to the unit hypersphere, is inherently more robust to low-precision arithmetic.<n>We validate this approach on both a 1.2B dense model and hybrid (Mamba-Transformer) MoE models of up to 3B/30B parameters.
  arXiv  Detail & Related papers  (2026-05-07T11:54:07Z)
• Hybrid Gated Flow (HGF): Stabilizing 1.58-bit LLMs via Selective Low-Rank Correction [0.766310831583367]
  Hybrid Gated Flow (HGF) is a dual-stream architecture that couples a 1.58-bit ternary backbone with a learnable, low-rank FP16 correction path.<n>We show that HGF 5.4 achieves a validation loss of 0.9306 compared to BitNet's 1.0294, recovering approximately 55% of the quality gap between pure ternary quantization and the FP16 baseline.
  arXiv  Detail & Related papers  (2026-02-05T03:47:17Z)
• Privacy-Enhancing Infant Cry Classification with Federated Transformers and Denoising Regularization [0.0]
  We present an end-to-end infant cry analysis pipeline that integrates a denoising autoencoder, a convolutional tokenizer, and a Transformer encoder.<n>The system performs on-device denoising, adaptive segmentation, post hoc calibration, and energy-based out-of-distribution (OOD) abstention.<n>Using the Baby Chillanto and Donate-a-Cry datasets, the model achieves a macro F1 score of 0.938, an AUC with 0.962, and an Expected Error (ECE) of 0.032, while reducing per-round client upload from approximately 36 to 42 MB to 3.3 MB
  arXiv  Detail & Related papers  (2025-12-15T20:33:24Z)
• Extreme Model Compression for Edge Vision-Language Models: Sparse Temporal Token Fusion and Adaptive Neural Compression [0.0]
  Two adaptive compression techniques are proposed to integrate algorithmic innovations with hardware-aware optimizations.<n>On event-based vision tasks, STTF reduces average token count by 84%.<n>ANC cuts FLOPs by up to 90% in low-motion scenes.
  arXiv  Detail & Related papers  (2025-11-23T15:43:00Z)
• Physics-informed GNN for medium-high voltage AC power flow with edge-aware attention and line search correction operator [7.927606793404479]
  Physics-informed graph neural networks (PIGNNs) have emerged as fast AC power-flow solvers.<n>PIGNN-Attn-LS achieves a test RMSE of 0.00033 p.u. in voltage and 0.08$circ$ in angle, outperforming the PIGNN-MLP baseline by 99.5% and 87.1%, respectively.
  arXiv  Detail & Related papers  (2025-09-26T15:09:26Z)
• Advanced Deep Learning Techniques for Automated Segmentation of Type B Aortic Dissections [4.545298205355719]
  We developed four deep learning-based pipelines for Type B aortic dissection segmentation.<n>Our approach achieved superior segmentation accuracy, with Dice Coefficients of 0.91 $pm$ 0.07 for TL, 0.88 $pm$ 0.18 for FL, and 0.47 $pm$ 0.25 for.
  arXiv  Detail & Related papers  (2025-06-27T13:38:33Z)
• CEReBrO: Compact Encoder for Representations of Brain Oscillations Using Efficient Alternating Attention [46.47343031985037]
  We introduce a Compact for Representations of Brain Oscillations using alternating attention (CEReBrO)<n>Our tokenization scheme represents EEG signals at a per-channel patch.<n>We propose an alternating attention mechanism that jointly models intra-channel temporal dynamics and inter-channel spatial correlations, achieving 2x speed improvement with 6x less memory required compared to standard self-attention.
  arXiv  Detail & Related papers  (2025-01-18T21:44:38Z)
• Data-Free Dynamic Compression of CNNs for Tractable Efficiency [46.498278084317704]
  structured pruning approaches have shown promise in lowering floating-point operations without substantial drops in accuracy.<n>We propose HASTE (Hashing for Tractable Efficiency), a data-free, plug-and-play convolution module that instantly reduces a network's test-time inference cost without training or fine-tuning.<n>We demonstrate our approach on the popular vision benchmarks CIFAR-10 and ImageNet, where we achieve a 46.72% reduction in FLOPs with only a 1.25% loss in accuracy.
  arXiv  Detail & Related papers  (2023-09-29T13:09:40Z)
• GDP: Stabilized Neural Network Pruning via Gates with Differentiable Polarization [84.57695474130273]
  Gate-based or importance-based pruning methods aim to remove channels whose importance is smallest. GDP can be plugged before convolutional layers without bells and whistles, to control the on-and-off of each channel. Experiments conducted over CIFAR-10 and ImageNet datasets show that the proposed GDP achieves the state-of-the-art performance.
  arXiv  Detail & Related papers  (2021-09-06T03:17:10Z)
• Towards a Competitive End-to-End Speech Recognition for CHiME-6 Dinner Party Transcription [73.66530509749305]
  In this paper, we argue that, even in difficult cases, some end-to-end approaches show performance close to the hybrid baseline. We experimentally compare and analyze CTC-Attention versus RNN-Transducer approaches along with RNN versus Transformer architectures. Our best end-to-end model based on RNN-Transducer, together with improved beam search, reaches quality by only 3.8% WER abs. worse than the LF-MMI TDNN-F CHiME-6 Challenge baseline.
  arXiv  Detail & Related papers  (2020-04-22T19:08:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.