FEMBA on the Edge: Physiologically-Aware Pre-Training, Quantization, and Deployment of a Bidirectional Mamba EEG Foundation Model on an Ultra-low Power Microcontroller

• URL: http://arxiv.org/abs/2603.26716v1
• Date: Wed, 18 Mar 2026 13:23:13 GMT
• Title: FEMBA on the Edge: Physiologically-Aware Pre-Training, Quantization, and Deployment of a Bidirectional Mamba EEG Foundation Model on an Ultra-low Power Microcontroller
• Abstract summary: We present FEMBA, a bidirectional Mamba architecture pre-trained on over 21,000 hours of EEG.<n>We introduce a novel Physiologically-Aware pre-training objective, consisting of a reconstruction with low-pass filtering.<n>We employ Quantization-Aware Training (QAT) to compress the model to 2-bit weights.
• Score: 13.961569964076451
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Objective: To enable continuous, long-term neuro-monitoring on wearable devices by overcoming the computational bottlenecks of Transformer-based Electroencephalography (EEG) foundation models and the quantization challenges inherent to State-Space Models (SSMs). Methods: We present FEMBA, a bidirectional Mamba architecture pre-trained on over 21,000 hours of EEG. We introduce a novel Physiologically-Aware pre-training objective, consisting of a reconstruction with low-pass filtering, to prioritize neural oscillations over high-frequency artifacts. To address the activation outliers common in SSMs, we employ Quantization-Aware Training (QAT) to compress the model to 2-bit weights. The framework is deployed on a parallel ultra-low-power RISC-V microcontroller (GAP9) using a custom double-buffered memory streaming scheme. Results: The proposed low-pass pre-training improves downstream AUROC on TUAB from 0.863 to 0.893 and AUPR from 0.862 to 0.898 compared to the best contrastive baseline. QAT successfully compresses weights with negligible performance loss, whereas standard post-training quantization degrades accuracy by approximately \textbf{30\%}. The embedded implementation achieves deterministic real-time inference (\textbf{1.70~s} per 5~s window) and reduces the memory footprint by \textbf{74\%} (to $\approx$2~MB), achieving competitive accuracy with up to \textbf{27$\times$} fewer FLOPs than Transformer benchmarks. Conclusion: FEMBA demonstrates that Mamba-based foundation models can be effectively quantized and deployed on extreme-edge hardware without sacrificing the representation quality required for robust clinical analysis. Significance: This work establishes the first full-stack framework for deploying large-scale EEG foundation models on ultra-low-power wearables, facilitating continuous, SSM based monitoring for epilepsy and sleep disorders.

Related papers

• Large-Small Model Collaboration for Zero-Shot Surgical Phase Recognition [45.191945465591374]
  Task-specific lightweight models for surgical phase recognition excel at capturing temporal dynamics but generalize poorly under domain shift.<n>We propose textbfLarge-textbfSmall textbfTemporal adaptation (textbfLaST), a novel large-small collaborative framework that enables zero-shot adaptation to unseen clinical domains.
  arXiv  Detail & Related papers  (2026-08-24T07:06:52Z)
• QB-LIF: Learnable-Scale Quantized Burst Neurons for Efficient SNNs [18.0608697536269]
  Quantized Burst-LIF neuron reformulates burst spiking as a saturated uniform quantization of membrane potentials with a learnable scale.<n> QB-LIF consistently outperforms binary and fixed-burst SNNs, achieving higher accuracy under ultra-low latency.
  arXiv  Detail & Related papers  (2026-04-28T14:18:53Z)
• LuMamba: Latent Unified Mamba for Electrode Topology-Invariant and Efficient EEG Modeling [14.663411219312827]
  We propose textbfLuMamba (textbfLatent textbfUnified textbfMamba), a self-supervised framework combining topology-invariant encodings with linear-complexity state-space modeling.<n>With only 4.6M parameters, LuMamba attains 80.99% balanced accuracy on TUAB and achieves state-of-art performance on Alzheimer's detection.
  arXiv  Detail & Related papers  (2026-03-19T16:26:11Z)
• SCALE:Scalable Conditional Atlas-Level Endpoint transport for virtual cell perturbation prediction [46.645872209565574]
  We present a specialized large-scale foundation model for virtual cell perturbation prediction.<n>First, we build a BioNeMo-based training and inference framework that substantially improves data throughput.<n>Second, we formulate perturbation prediction as conditional transport and implement it with a set-aware flow architecture.<n>Third, we evaluate the model on Tahoe-100M using a rigorous cell-level protocol centered on biologically meaningful metrics.
  arXiv  Detail & Related papers  (2026-03-18T05:51:08Z)
• From Evaluation to Design: Using Potential Energy Surface Smoothness Metrics to Guide Machine Learning Interatomic Potential Architectures [12.68400434984463]
  MLIPs fail to reproduce the physical smoothness of the quantum potential energy surface.<n>Existing evaluations, such as microcanonical molecular dynamics, are computationally expensive and primarily probe near-equilibrium states.<n>We introduce the Bond Smoothness Characterization Test (BSCT) to improve evaluation metrics for MLIPs.
  arXiv  Detail & Related papers  (2026-02-04T18:50:10Z)
• ECO: Quantized Training without Full-Precision Master Weights [58.97082407934466]
  Error-Compensating (ECO) eliminates master weights by applying updates directly to quantized parameters.<n>We show that ECO converges to a constant-radius neighborhood of the optimum, while naive master-weight removal can incur an error that is inversely proportional to the learning rate.
  arXiv  Detail & Related papers  (2026-01-29T18:35:01Z)
• PT$^2$-LLM: Post-Training Ternarization for Large Language Models [52.4629647715623]
  Large Language Models (LLMs) have shown impressive capabilities across diverse tasks, but their large memory and compute demands hinder deployment.<n>We propose PT$2$-LLM, a post-training ternarization framework tailored for LLMs.<n>At its core is an Asymmetric Ternary Quantizer equipped with a two-stage refinement pipeline.
  arXiv  Detail & Related papers  (2025-09-27T03:01:48Z)
• MPQ-DMv2: Flexible Residual Mixed Precision Quantization for Low-Bit Diffusion Models with Temporal Distillation [74.34220141721231]
  We present MPQ-DMv2, an improved textbfMixed textbfPrecision textbfQuantization framework for extremely low-bit textbfDiffusion textbfModels.
  arXiv  Detail & Related papers  (2025-07-06T08:16:50Z)
• Adaptive Gate-Aware Mamba Networks for Magnetic Resonance Fingerprinting [8.673589204148115]
  Magnetic Resonance Fingerprinting (MRF) enables fast quantitative imaging by matching signal evolutions to a predefined dictionary.<n>Recent work has explored deep learning-based approaches as alternatives to DM.<n>We propose GAST-Mamba, an end-to-end framework that combines a dual Mamba-based encoder with a Gate-Aware Spatial-Temporal (GAST) processor.
  arXiv  Detail & Related papers  (2025-07-04T08:04:01Z)
• FIMA-Q: Post-Training Quantization for Vision Transformers by Fisher Information Matrix Approximation [55.12070409045766]
  Post-training quantization (PTQ) has stood out as a cost-effective and promising model compression paradigm in recent years.<n>Current PTQ methods for Vision Transformers (ViTs) still suffer from significant accuracy degradation, especially under low-bit quantization.
  arXiv  Detail & Related papers  (2025-06-13T07:57:38Z)
• FEMBA: Efficient and Scalable EEG Analysis with a Bidirectional Mamba Foundation Model [14.663411219312827]
  We present FEMBA (Foundational EEG Mamba + Bidirectional Architecture), a novel self-supervised framework for EEG analysis.<n>Unlike Transformer-based models, which incur quadratic time and memory complexity, FEMBA scales linearly with sequence length.<n>It achieves competitive performance in comparison with transformer models, with significantly lower computational cost.
  arXiv  Detail & Related papers  (2025-02-10T13:15:52Z)
• A2Q: Accumulator-Aware Quantization with Guaranteed Overflow Avoidance [49.1574468325115]
  accumulator-aware quantization (A2Q) is a novel weight quantization method designed to train quantized neural networks (QNNs) to avoid overflow during inference. A2Q introduces a unique formulation inspired by weight normalization that constrains the L1-norm of model weights according to accumulator bit width bounds. We show A2Q can train QNNs for low-precision accumulators while maintaining model accuracy competitive with a floating-point baseline.
  arXiv  Detail & Related papers  (2023-08-25T17:28:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.