BrainVista: Modeling Naturalistic Brain Dynamics as Multimodal Next-Token Prediction

• URL: http://arxiv.org/abs/2602.04512v1
• Date: Wed, 04 Feb 2026 13:00:06 GMT
• Title: BrainVista: Modeling Naturalistic Brain Dynamics as Multimodal Next-Token Prediction
• Abstract summary: We introduce BrainVista, a multimodal autoregressive framework to model the causal evolution of brain states.<n>BrainVista incorporates Network-wise Tokenizers to disentangle system-specific dynamics and a Spatial Mixer Head that captures inter-network information flow.<n>We validate our framework on Algonauts 2025, CineBrain, and HAD, achieving state-of-the-art fMRI encoding performance.
• Score: 20.44304015989378
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Naturalistic fMRI characterizes the brain as a dynamic predictive engine driven by continuous sensory streams. However, modeling the causal forward evolution in realistic neural simulation is impeded by the timescale mismatch between multimodal inputs and the complex topology of cortical networks. To address these challenges, we introduce BrainVista, a multimodal autoregressive framework designed to model the causal evolution of brain states. BrainVista incorporates Network-wise Tokenizers to disentangle system-specific dynamics and a Spatial Mixer Head that captures inter-network information flow without compromising functional boundaries. Furthermore, we propose a novel Stimulus-to-Brain (S2B) masking mechanism to synchronize high-frequency sensory stimuli with hemodynamically filtered signals, enabling strict, history-only causal conditioning. We validate our framework on Algonauts 2025, CineBrain, and HAD, achieving state-of-the-art fMRI encoding performance. In long-horizon rollout settings, our model yields substantial improvements over baselines, increasing pattern correlation by 36.0\% and 33.3\% on relative to the strongest baseline Algonauts 2025 and CineBrain, respectively.

Related papers

• Self-Supervised Consistency Enhanced Disentangled Learning for Neural Decoding Generalization in Brain-Machine Interface [12.519778500895617]
  Brain-Machine Interfaces (BMIs) provide a direct communication pathway between the brain and external devices, with potential applications in rehabilitation, human motor augmentation, and human-centered robotics.<n>Existing solutions suffer from two main drawbacks: difficulty in learning robust neural representations, and neglecting that neural drift varies across motor parameters.<n>We propose Self-Supervised Consistency enhanced Disentangled Learning (SSCDL), a neural decoding generalization framework built on two key innovations.
  arXiv  Detail & Related papers  (2026-07-27T05:40:01Z)
• BrainDyn: A Sheaf Neural ODE for Generative Brain Dynamics [8.837573450847225]
  We introduce BrainDyn, a sheaf neural ordinary differential equation (neural ODE) model for continuous-time dynamics on structured brain graphs.<n>BrainDyn encodes the recent activity history of each brain region using a long short-term memory (LSTM) model.<n>The resulting representations support downstream tasks including in silico perturbation prediction.
  arXiv  Detail & Related papers  (2026-05-19T04:03:01Z)
• Brain-Inspired Capture: Evidence-Driven Neuromimetic Perceptual Simulation for Visual Decoding [28.75188553928776]
  Brain-Inspired Capture (BI-Cap) is a neuromimetic perceptual simulation paradigm that aligns neural and visual modalities.<n>BI-Cap substantially outperforms state-of-the-art methods, achieving relative gains of 9.2% and 8.0%, respectively.
  arXiv  Detail & Related papers  (2026-04-20T08:07:33Z)
• Brain-WM: Brain Glioblastoma World Model [56.45393372490125]
  We present Brain-WM, a pioneering GBM world model that unifies next-step treatment prediction and future MRI generation.<n>Brain-WM encodes latent dynamics into a shared latent space for joint autoregressive treatment prediction and flow-based future MRI generation.<n>Ultimately, Brain-WM offers a robust clinical sandbox for optimizing patient healthcare.
  arXiv  Detail & Related papers  (2026-03-08T09:54:08Z)
• BrainFuse: a unified infrastructure integrating realistic biological modeling and core AI methodology [34.597782096704115]
  Neuroscience and artificial intelligence represent distinct yet complementary pathways to general intelligence.<n>BrainFuse is a unified infrastructure that provides comprehensive support for biophysical neural simulation and gradient-based learning.<n>BrainFuse serves as a foundational engine to facilitate cross-disciplinary research and accelerate the development of next-generation bio-inspired intelligent systems.
  arXiv  Detail & Related papers  (2026-01-29T08:44:01Z)
• BrainATCL: Adaptive Temporal Brain Connectivity Learning for Functional Link Prediction and Age Estimation [0.33748750222488655]
  We propose BrainATCL, an unsupervised, nonparametric framework for adaptive temporal brain connectivity learning.<n>Our method dynamically adjusts the lookback window for each snapshot based on the rate of newly added edges.<n>Graph sequences are encoded using a GINE-Mamba2 backbone to learn spatial-temporal representations of dynamic functional connectivity in resting-state fMRI data.
  arXiv  Detail & Related papers  (2025-08-09T21:18:25Z)
• TRIBE: TRImodal Brain Encoder for whole-brain fMRI response prediction [7.864304771129752]
  TRIBE is the first deep neural network trained to predict brain responses to stimuli across multiple modalities.<n>Our model can precisely model the spatial and temporal fMRI responses to videos.<n>Our approach paves the way towards building an integrative model of representations in the human brain.
  arXiv  Detail & Related papers  (2025-07-29T20:52:31Z)
• Langevin Flows for Modeling Neural Latent Dynamics [81.81271685018284]
  We introduce LangevinFlow, a sequential Variational Auto-Encoder where the time evolution of latent variables is governed by the underdamped Langevin equation.<n>Our approach incorporates physical priors -- such as inertia, damping, a learned potential function, and forces -- to represent both autonomous and non-autonomous processes in neural systems.<n>Our method outperforms state-of-the-art baselines on synthetic neural populations generated by a Lorenz attractor.
  arXiv  Detail & Related papers  (2025-07-15T17:57:48Z)
• Voxel-Level Brain States Prediction Using Swin Transformer [65.9194533414066]
  We propose a novel architecture which employs a 4D Shifted Window (Swin) Transformer as encoder to efficiently learn-temporal information and a convolutional decoder to enable brain state prediction at the same spatial and temporal resolution as the input fMRI data.<n>Our model has shown high accuracy when predicting 7.2s resting-state brain activities based on the prior 23.04s fMRI time series.<n>This shows promising evidence that thetemporal organization of the human brain can be learned by a Swin Transformer model, at high resolution, which provides a potential for reducing fMRI scan time and the development of brain-computer interfaces
  arXiv  Detail & Related papers  (2025-06-13T04:14:38Z)
• Spatiotemporal Learning of Brain Dynamics from fMRI Using Frequency-Specific Multi-Band Attention for Cognitive and Psychiatric Applications [5.199807441687141]
  Multi-Band Net Brain (MBBN) is the first transformer-based framework to explicitly model frequency-specific brain dynamics.<n>Training on 49,673 individuals across three large-scale cohorts, MBBN sets a new state-of-the-art in predicting psychiatric and cognitive outcomes.
  arXiv  Detail & Related papers  (2025-03-30T10:56:50Z)
• Neural Manifolds and Cognitive Consistency: A New Approach to Memory Consolidation in Artificial Systems [0.0]
  We introduce a novel mathematical framework that unifies neural population dynamics, hippocampal sharp wave-ripple (SpWR) generation, and cognitive consistency constraints inspired by Heider's theory.<n>Our model leverages low-dimensional manifold representations to capture structured neural drift and incorporates a balance energy function to enforce coherent synaptic interactions.<n>This work paves the way for scalable neuromorphic architectures that bridge neuroscience and artificial intelligence, offering more robust and adaptive learning mechanisms for future intelligent systems.
  arXiv  Detail & Related papers  (2025-02-25T18:28:25Z)
• Artificial Kuramoto Oscillatory Neurons [65.16453738828672]
  It has long been known in both neuroscience and AI that ''binding'' between neurons leads to a form of competitive learning where representations are compressed in order to represent more abstract concepts in deeper layers of the network.<n>We introduce Artificial rethinking together with arbitrary connectivity designs such as fully connected convolutional, or attentive mechanisms.<n>We show that this idea provides performance improvements across a wide spectrum of tasks such as unsupervised object discovery, adversarial robustness, uncertainty, quantification, and reasoning.
  arXiv  Detail & Related papers  (2024-10-17T17:47:54Z)
• Contrastive Learning in Memristor-based Neuromorphic Systems [55.11642177631929]
  Spiking neural networks have become an important family of neuron-based models that sidestep many of the key limitations facing modern-day backpropagation-trained deep networks. In this work, we design and investigate a proof-of-concept instantiation of contrastive-signal-dependent plasticity (CSDP), a neuromorphic form of forward-forward-based, backpropagation-free learning.
  arXiv  Detail & Related papers  (2024-09-17T04:48:45Z)
• Interpretable Spatio-Temporal Embedding for Brain Structural-Effective Network with Ordinary Differential Equation [56.34634121544929]
  In this study, we first construct the brain-effective network via the dynamic causal model. We then introduce an interpretable graph learning framework termed Spatio-Temporal Embedding ODE (STE-ODE) This framework incorporates specifically designed directed node embedding layers, aiming at capturing the dynamic interplay between structural and effective networks.
  arXiv  Detail & Related papers  (2024-05-21T20:37:07Z)
• BrainODE: Dynamic Brain Signal Analysis via Graph-Aided Neural Ordinary Differential Equations [67.79256149583108]
  We propose a novel model called BrainODE to achieve continuous modeling of dynamic brain signals. By learning latent initial values and neural ODE functions from irregular time series, BrainODE effectively reconstructs brain signals at any time point.
  arXiv  Detail & Related papers  (2024-04-30T10:53:30Z)
• MindBridge: A Cross-Subject Brain Decoding Framework [60.58552697067837]
  Brain decoding aims to reconstruct stimuli from acquired brain signals. Currently, brain decoding is confined to a per-subject-per-model paradigm. We present MindBridge, that achieves cross-subject brain decoding by employing only one model.
  arXiv  Detail & Related papers  (2024-04-11T15:46:42Z)
• Ranking of Communities in Multiplex Spatiotemporal Models of Brain Dynamics [0.0]
  We propose an interpretation of neural HMMs as multiplex brain state graph models we term Hidden Markov Graph Models (HMs) This interpretation allows for dynamic brain activity to be analysed using the full repertoire of network analysis techniques. We produce a new tool for determining important communities of brain regions using a random walk-based procedure.
  arXiv  Detail & Related papers  (2022-03-17T12:14:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.