BrainWideBench: Benchmarking large-scale pretraining and across-animal transfer in multi-region neural recordings
Abstract Overview
BrainWideBench is a benchmark for evaluating across-animal transfer in multi-region neural recordings, built on the International Brain Laboratory Brainwide Map dataset. The benchmark uses recordings from 139 mice spanning 276 brain regions and organizes evaluation into three task suites: behavior prediction, neural activity prediction, and neuron identity/brain region prediction. It enforces animal-level separation between pretraining and evaluation, with models pretrained on 126 mice and tested on 13 held-out mice under finetuning and zero-shot regimes. Across these suites, the paper studies whether large-scale pretraining yields neural representations that generalize across animals, sessions, and downstream tasks.
Novelty
The paper’s main novelty is a unified, reproducible benchmark for large-scale pretraining and across-animal transfer in brain-wide neural recordings, rather than evaluating models on isolated datasets or single task families. Its distinctive contribution is the combination of standardized splits, quality-control metadata, and three complementary task suites that jointly probe behavioral, dynamical, and anatomical structure.
Results
The benchmark shows that pretraining generally improves transfer relative to matched single-session baselines, but gains depend strongly on how well the pretraining objective matches the downstream task. In behavior decoding, POYO+ and NDT-Stitch improve over their non-pretrained counterparts in average rank (4.80 to 2.53 for POYO/POYO+ and 6.80 to 3.58 for NDT/NDT-Stitch); in neural activity prediction, NDT-Stitch is strongest on forecasting while MtM is strongest on co-smoothing. For brain region prediction, specialized neuron-identity models perform best, with NuCLR reaching macro-F1 0.654 in the multi-unit linear-probe setting, indicating that no single method dominates all three suites.
Key Points
- BrainWideBench evaluates across-animal transfer on the IBL Brainwide Map using standardized animal-level splits, quality-control metadata, and three task suites covering behavior, neural dynamics, and anatomy.
- Pretraining improves performance in behavior decoding and neural activity prediction, but transfer is heterogeneous and strongly shaped by alignment between pretraining objectives and evaluation tasks.
- Brain region prediction is best handled by methods designed for neuron-level identity representations, with NuCLR and NEMO outperforming models pretrained primarily for decoding or masked neural prediction.