Advancing Brainwave Modeling with a Codebook-Based Foundation Model

• URL: http://arxiv.org/abs/2505.16724v1
• Date: Thu, 22 May 2025 14:32:56 GMT
• Title: Advancing Brainwave Modeling with a Codebook-Based Foundation Model
• Abstract summary: We introduce LaBraM++, an enhanced Large Brainwave Foundation Model (LBM) that incorporates principled improvements grounded in robust signal processing foundations.<n>LaBraM++ demonstrates substantial gains across a variety of tasks, consistently outperforming its originally-based architecture and achieving competitive results when compared to other open-source LBMs.
• Score: 41.525984326072596
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances in large-scale pre-trained Electroencephalogram (EEG) models have shown great promise, driving progress in Brain-Computer Interfaces (BCIs) and healthcare applications. However, despite their success, many existing pre-trained models have struggled to fully capture the rich information content of neural oscillations, a limitation that fundamentally constrains their performance and generalizability across diverse BCI tasks. This limitation is frequently rooted in suboptimal architectural design choices which constrain their representational capacity. In this work, we introduce LaBraM++, an enhanced Large Brainwave Foundation Model (LBM) that incorporates principled improvements grounded in robust signal processing foundations. LaBraM++ demonstrates substantial gains across a variety of tasks, consistently outperforming its originally-based architecture and achieving competitive results when compared to other open-source LBMs. Its superior performance and training efficiency highlight its potential as a strong foundation for future advancements in LBMs.

Related papers

• SkillMemo: Expert-guided Skill Memory Framework for Compositional Embodied Manipulation [72.14179740056073]
  Embodied visuomotor models, including Diffusion Policy (DP) and Vision-Language-Action (VLA) models, have demonstrated promising performance on robotic manipulation benchmarks.<n>We propose Skill-Based Memory (SkillMemo) framework that implicitly decomposes long-horizon demonstrations into latent atomic skills.
  arXiv  Detail & Related papers  (2026-08-06T12:46:13Z)
• On Predicting the Post-training Potential of Pre-trained LLMs [60.07459271263409]
  We introduce a new task of predicting post-training potential - forecasting a base model's performance before post-training.<n>We propose RuDE, a unified framework that bypasses the generation gap of base models by leveraging response discrimination.<n>Experiments demonstrate a correlation greater than 90% with post-training performance.
  arXiv  Detail & Related papers  (2026-05-12T11:33:49Z)
• DeepThinkVLA: Enhancing Reasoning Capability of Vision-Language-Action Models [51.76664843721462]
  DeepThinkVLA is a new architecture for Vision-Language-Action models.<n>It generates sequential CoT with causal attention and switches to bidirectional attention for fast decoding of action vectors.<n>It achieves a 97.0% success rate on the LIBERO benchmark.
  arXiv  Detail & Related papers  (2025-10-31T05:26:16Z)
• Towards Stable and Effective Reinforcement Learning for Mixture-of-Experts [113.0656076371565]
  We propose a novel router-aware approach to optimize importance sampling weights in off-policy reinforcement learning (RL)<n>Specifically, we design a rescaling strategy guided by router logits, which effectively reduces gradient variance and mitigates training divergence.<n> Experimental results demonstrate that our method significantly improves both the convergence stability and the final performance of MoE models.
  arXiv  Detail & Related papers  (2025-10-27T05:47:48Z)
• SpikingBrain: Spiking Brain-inspired Large Models [42.41339012839023]
  SpikingBrain is a family of brain-inspired models for efficient long-context training and inference.<n>We develop two models: SpikingBrain-7B, a linear LLM, and SpikingBrain-76B, a hybrid-linear MoE LLM.<n>Our models significantly improve long-sequence training efficiency and deliver inference with (partially) constant memory and event-driven spiking behavior.
  arXiv  Detail & Related papers  (2025-09-05T17:34:00Z)
• Are Large Brainwave Foundation Models Capable Yet? Insights from Fine-tuning [41.40603531008809]
  We evaluate current Large Brainwave Foundation Models (LBMs) through systematic fine-tuning experiments.<n>Our analysis shows that state-of-the-art LBMs achieve only marginal improvements (0.9%-1.2%) over traditional deep architectures.
  arXiv  Detail & Related papers  (2025-07-01T21:21:42Z)
• FUDOKI: Discrete Flow-based Unified Understanding and Generation via Kinetic-Optimal Velocities [76.46448367752944]
  multimodal large language models (MLLMs) unify visual understanding and image generation within a single framework.<n>Most existing MLLMs rely on autore (AR) architectures, which impose inherent limitations on future development.<n>We introduce FUDOKI, a unified multimodal model purely based on discrete flow matching.
  arXiv  Detail & Related papers  (2025-05-26T15:46:53Z)
• Scaling Laws for Native Multimodal Models [53.490942903659565]
  We revisit the architectural design of native multimodal models and conduct an extensive scaling laws study.<n>Our investigation reveals no inherent advantage to late-fusion architectures over early-fusion ones.<n>We show that incorporating Mixture of Experts (MoEs) allows for models that learn modality-specific weights, significantly enhancing performance.
  arXiv  Detail & Related papers  (2025-04-10T17:57:28Z)
• OpenVLThinker: An Early Exploration to Complex Vision-Language Reasoning via Iterative Self-Improvement [91.88062410741833]
  This study investigates whether similar reasoning capabilities can be successfully integrated into large vision-language models (LVLMs)<n>We consider an approach that iteratively leverages supervised fine-tuning (SFT) on lightweight training data and Reinforcement Learning (RL) to further improve model generalization.<n>OpenVLThinker, a LVLM exhibiting consistently improved reasoning performance on challenging benchmarks such as MathVista, MathVerse, and MathVision, demonstrates the potential of our strategy for robust vision-language reasoning.
  arXiv  Detail & Related papers  (2025-03-21T17:52:43Z)
• Will Pre-Training Ever End? A First Step Toward Next-Generation Foundation MLLMs via Self-Improving Systematic Cognition [86.21199607040147]
  Self-Improving cognition (SIcog) is a self-learning framework for constructing next-generation foundation language models.<n>We introduce Chain-of-Description, a step-by-step visual understanding method, and integrate structured chain-of-thought (CoT) reasoning to support in-depth multimodal reasoning.<n>Extensive experiments demonstrate that SIcog produces next-generation foundation MLLMs with substantially improved multimodal cognition.
  arXiv  Detail & Related papers  (2025-03-16T00:25:13Z)
• HM-DF SNN: Transcending Conventional Online Learning with Advanced Training and Deployment [39.6783548791379]
  Spiking Neural Networks (SNNs) are considered to have enormous potential in the future development of Artificial Intelligence.<n>Current online learning framework cannot tackle the inseparability problem of temporal dependent gradients.<n>We propose Hybrid Mechanism-Driven Firing (HM-DF) model, which is a family of advanced models that respectively adopt different spiking calculation schemes.
  arXiv  Detail & Related papers  (2024-10-10T02:39:22Z)
• On the Modeling Capabilities of Large Language Models for Sequential Decision Making [52.128546842746246]
  Large pretrained models are showing increasingly better performance in reasoning and planning tasks. We evaluate their ability to produce decision-making policies, either directly, by generating actions, or indirectly. In environments with unfamiliar dynamics, we explore how fine-tuning LLMs with synthetic data can significantly improve their reward modeling capabilities.
  arXiv  Detail & Related papers  (2024-10-08T03:12:57Z)
• Diffusion-Based Neural Network Weights Generation [80.89706112736353]
  D2NWG is a diffusion-based neural network weights generation technique that efficiently produces high-performing weights for transfer learning. Our method extends generative hyper-representation learning to recast the latent diffusion paradigm for neural network weights generation. Our approach is scalable to large architectures such as large language models (LLMs), overcoming the limitations of current parameter generation techniques.
  arXiv  Detail & Related papers  (2024-02-28T08:34:23Z)
• Dyna-bAbI: unlocking bAbI's potential with dynamic synthetic benchmarking [16.109330335379962]
  Dyna-bAbI is a dynamic framework providing fine-grained control over task generation in bAbI. We demonstrate our ideas by constructing three new tasks requiring compositional generalization.
  arXiv  Detail & Related papers  (2021-11-30T20:36:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.