Brain-Semantoks: Learning Semantic Tokens of Brain Dynamics with a Self-Distilled Foundation Model

• URL: http://arxiv.org/abs/2512.11582v1
• Date: Fri, 12 Dec 2025 14:11:20 GMT
• Title: Brain-Semantoks: Learning Semantic Tokens of Brain Dynamics with a Self-Distilled Foundation Model
• Authors: Sam Gijsen, Marc-Andre Schulz, Kerstin Ritter,
• Abstract summary: We introduce Brain-Semantoks, a self-supervised framework to learn abstract representations of brain dynamics.<n>Its architecture is built on two core innovations: a semantic tokenizer that aggregates noisy regional signals into robust tokens representing functional networks.<n>We show that learned representations enable strong performance on a variety of downstream tasks even when only using a linear probe.
• Score: 0.27528170226206433
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The development of foundation models for functional magnetic resonance imaging (fMRI) time series holds significant promise for predicting phenotypes related to disease and cognition. Current models, however, are often trained using a mask-and-reconstruct objective on small brain regions. This focus on low-level information leads to representations that are sensitive to noise and temporal fluctuations, necessitating extensive fine-tuning for downstream tasks. We introduce Brain-Semantoks, a self-supervised framework designed specifically to learn abstract representations of brain dynamics. Its architecture is built on two core innovations: a semantic tokenizer that aggregates noisy regional signals into robust tokens representing functional networks, and a self-distillation objective that enforces representational stability across time. We show that this objective is stabilized through a novel training curriculum, ensuring the model robustly learns meaningful features from low signal-to-noise time series. We demonstrate that learned representations enable strong performance on a variety of downstream tasks even when only using a linear probe. Furthermore, we provide comprehensive scaling analyses indicating more unlabeled data reliably results in out-of-distribution performance gains without domain adaptation.

Related papers

• BrainRVQ: A High-Fidelity EEG Foundation Model via Dual-Domain Residual Quantization and Hierarchical Autoregression [26.114257185901838]
  We propose BrainRVQ, a general-purpose EEG foundation model pre-trained on a large-scale corpus of clinical EEG data.<n>BrainRVQ features a Dual-Domain Residual Vector Quantization (DD-RVQ) tokenizer that disentangles temporal waveforms and spectral patterns into hierarchical discrete codes.
  arXiv  Detail & Related papers  (2026-02-18T23:30:36Z)
• Self-Supervised Learning via Flow-Guided Neural Operator on Time-Series Data [57.85958428020496]
  Flow-Guided Neural Operator (FGNO) is a novel framework combining operator learning with flow matching for SSL training.<n>FGNO learns mappings in functional spaces by using Short-Time Fourier Transform to unify different time resolutions.<n>Unlike prior generative SSL methods that use noisy inputs during inference, we propose using clean inputs for representation extraction while learning representations with noise.
  arXiv  Detail & Related papers  (2026-02-12T18:54:57Z)
• Simple Models, Rich Representations: Visual Decoding from Primate Intracortical Neural Signals [0.0]
  We address the problem of decoding visual information from high-density intracortical recordings in primates.<n>We develop a modular generative decoding pipeline that combines low-resolution latent reconstruction with semantically conditioned diffusion.<n>This framework provides principles for brain-computer interfaces and semantic neural decoding.
  arXiv  Detail & Related papers  (2026-01-16T09:10:31Z)
• A Novel Framework for Learning Stochastic Representations for Sequence Generation and Recognition [0.0]
  The ability to generate and recognize sequential data is fundamental for autonomous systems operating in dynamic environments.<n>We propose a novel Recurrent Network with Parametric Biases (RNNPB)<n>Our approach provides a framework for modeling temporal patterns and advances the development of robust and systems in artificial intelligence and robotics.
  arXiv  Detail & Related papers  (2024-12-30T07:27:50Z)
• BrainMAE: A Region-aware Self-supervised Learning Framework for Brain Signals [11.030708270737964]
  We propose Brain Masked Auto-Encoder (BrainMAE) for learning representations directly from fMRI time-series data. BrainMAE consistently outperforms established baseline methods by significant margins in four distinct downstream tasks.
  arXiv  Detail & Related papers  (2024-06-24T19:16:24Z)
• TokenUnify: Scaling Up Autoregressive Pretraining for Neuron Segmentation [65.65530016765615]
  We propose a hierarchical predictive coding framework that captures multi-scale dependencies through three complementary learning objectives.<n> TokenUnify integrates random token prediction, next-token prediction, and next-all token prediction to create a comprehensive representational space.<n>We also introduce a large-scale EM dataset with 1.2 billion annotated voxels, offering ideal long-sequence visual data with spatial continuity.
  arXiv  Detail & Related papers  (2024-05-27T05:45:51Z)
• DSAM: A Deep Learning Framework for Analyzing Temporal and Spatial Dynamics in Brain Networks [4.041732967881764]
  Most rs-fMRI studies compute a single static functional connectivity matrix across brain regions of interest. These approaches are at risk of oversimplifying brain dynamics and lack proper consideration of the goal at hand. We propose a novel interpretable deep learning framework that learns goal-specific functional connectivity matrix directly from time series.
  arXiv  Detail & Related papers  (2024-05-19T23:35:06Z)
• Understanding Activation Patterns in Artificial Neural Networks by Exploring Stochastic Processes [0.0]
  We propose utilizing the framework of processes, which has been underutilized thus far. We focus solely on activation frequency, leveraging neuroscience techniques used for real neuron spike trains. We derive parameters describing activation patterns in each network, revealing consistent differences across architectures and training sets.
  arXiv  Detail & Related papers  (2023-08-01T22:12:30Z)
• Joint fMRI Decoding and Encoding with Latent Embedding Alignment [77.66508125297754]
  We introduce a unified framework that addresses both fMRI decoding and encoding. Our model concurrently recovers visual stimuli from fMRI signals and predicts brain activity from images within a unified framework.
  arXiv  Detail & Related papers  (2023-03-26T14:14:58Z)
• Understanding Self-Predictive Learning for Reinforcement Learning [61.62067048348786]
  We study the learning dynamics of self-predictive learning for reinforcement learning. We propose a novel self-predictive algorithm that learns two representations simultaneously.
  arXiv  Detail & Related papers  (2022-12-06T20:43:37Z)
• Deep Representations for Time-varying Brain Datasets [4.129225533930966]
  This paper builds an efficient graph neural network model that incorporates both region-mapped fMRI sequences and structural connectivities as inputs. We find good representations of the latent brain dynamics through learning sample-level adaptive adjacency matrices. These modules can be easily adapted to and are potentially useful for other applications outside the neuroscience domain.
  arXiv  Detail & Related papers  (2022-05-23T21:57:31Z)
• Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
  We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs. By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
  arXiv  Detail & Related papers  (2022-02-01T17:11:13Z)
• PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning [109.84770951839289]
  We present PredRNN, a new recurrent network for learning visual dynamics from historical context. We show that our approach obtains highly competitive results on three standard datasets.
  arXiv  Detail & Related papers  (2021-03-17T08:28:30Z)
• Automatic Recall Machines: Internal Replay, Continual Learning and the Brain [104.38824285741248]
  Replay in neural networks involves training on sequential data with memorized samples, which counteracts forgetting of previous behavior caused by non-stationarity. We present a method where these auxiliary samples are generated on the fly, given only the model that is being trained for the assessed objective. Instead the implicit memory of learned samples within the assessed model itself is exploited.
  arXiv  Detail & Related papers  (2020-06-22T15:07:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.