RABBiT: Rapidly adaptive BOLD foundation model via brain-tuning for accurate zero-shot and few-shot prediction of speech-elicited responses in the brain

• URL: http://arxiv.org/abs/2607.05171v1
• Date: Mon, 06 Jul 2026 14:54:19 GMT
• Title: RABBiT: Rapidly adaptive BOLD foundation model via brain-tuning for accurate zero-shot and few-shot prediction of speech-elicited responses in the brain
• Abstract summary: We introduce RABBiT, a compact audio-to-fMRI encoder for accurate zero- and few-shot prediction.<n>RABBiT enables scalable population-level analyses of language in the human brain.
• Score: 9.911908326274581
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Language understanding in the brain is context-dependent, varying across experimental stimuli and individuals, which makes it difficult to build computational models that generalize across both. This calls for a foundation model of language-evoked brain activity that can capture shared structure while adapting efficiently to new participants and inputs. We introduce RABBiT (Rapidly Adaptive BOLD foundation model via BraIn-Tuning), a compact audio-to-fMRI encoder designed for accurate zero- and few-shot prediction. A comprehensive evaluation on 324 participants across multiple unseen fMRI datasets shows that RABBiT enables accurate zero-shot prediction of fMRI responses to natural speech across auditory and language-selective regions, surpassing the SOTA foundation model for fMRI and predictions based on group averages. With as little as 10 minutes of participant-specific data, RABBiT further improves performance via parameter-efficient tuning, substantially outperforming per-participant linear models. RABBiT's performance is driven by two key innovations: (1) learned region-specific attention, and (2) a decomposition of brain responses into shared and subject-specific components, combined with a brain-tuned speech backbone. In addition to supporting strong predictive accuracy, the structured, region-specific representations that RABBiT learns enable interpretability. By eliminating the need for extensive per-participant data and model fitting, RABBiT enables scalable population-level analyses of language in the human brain. We make the code available at https://github.com/bridge-ai-neuro/rabbit.

Related papers

• Heterogeneous Neural Predictivity from Language Models During Naturalistic Comprehension [0.6331422666055228]
  Language-model representations provide structured, high-dimensional annotations of naturalistic language stimuli.<n>Positive held-out prediction and gains over low-level baselines were widespread in source-level summaries.<n>These findings show that language-model-derived quantities can annotate neural activity during natural speech and text comprehension.
  arXiv  Detail & Related papers  (2026-06-25T11:08:49Z)
• Brain-tuning Improves Generalizability and Efficiency of Brain Alignment in Speech Models [9.911908326274581]
  We fine-tune pretrained speech language models to jointly predict fMRI responses from multiple participants.<n>Our method leads to 1) a 5-fold decrease in the amount of fMRI data needed to predict brain data from new participants, 2) up to a 50% increase in the overall brain alignment, and 3) strong generalization to new unseen datasets.
  arXiv  Detail & Related papers  (2025-10-24T14:42:19Z)
• Generative causal testing to bridge data-driven models and scientific theories in language neuroscience [82.995061475971]
  We present generative causal testing (GCT), a framework for generating concise explanations of language selectivity in the brain.<n>We show that GCT can dissect fine-grained differences between brain areas with similar functional selectivity.
  arXiv  Detail & Related papers  (2024-10-01T15:57:48Z)
• A Lesion-aware Edge-based Graph Neural Network for Predicting Language Ability in Patients with Post-stroke Aphasia [12.129896943547912]
  We propose a lesion-aware graph neural network (LEGNet) to predict language ability from resting-state fMRI (rs-fMRI) connectivity in patients with post-stroke aphasia. Our model integrates three components: an edge-based learning module that encodes functional connectivity between brain regions, a lesion encoding module, and a subgraph learning module.
  arXiv  Detail & Related papers  (2024-09-03T21:28:48Z)
• fMRI predictors based on language models of increasing complexity recover brain left lateralization [4.1618731507412505]
  We show that the left-right difference in brain correlation follows a scaling law with the number of parameters. This finding reconciles computational analyses of brain activity using large language models with the classic observation from aphasic patients showing left hemisphere dominance for language.
  arXiv  Detail & Related papers  (2024-05-28T09:24:52Z)
• In-Context Language Learning: Architectures and Algorithms [73.93205821154605]
  We study ICL through the lens of a new family of model problems we term in context language learning (ICLL) We evaluate a diverse set of neural sequence models on regular ICLL tasks.
  arXiv  Detail & Related papers  (2024-01-23T18:59:21Z)
• Scaling laws for language encoding models in fMRI [47.498241053872924]
  We tested whether larger open-source models are better at predicting brain responses recorded using fMRI. Similar logarithmic behavior was observed when scaling the size of the fMRI training set. These results suggest that increasing scale in both models and data will yield incredibly effective models of language processing in the brain.
  arXiv  Detail & Related papers  (2023-05-19T17:53:03Z)
• Neural Language Models are not Born Equal to Fit Brain Data, but Training Helps [75.84770193489639]
  We examine the impact of test loss, training corpus and model architecture on the prediction of functional Magnetic Resonance Imaging timecourses of participants listening to an audiobook. We find that untrained versions of each model already explain significant amount of signal in the brain by capturing similarity in brain responses across identical words. We suggest good practices for future studies aiming at explaining the human language system using neural language models.
  arXiv  Detail & Related papers  (2022-07-07T15:37:17Z)
• Dependency-based Mixture Language Models [53.152011258252315]
  We introduce the Dependency-based Mixture Language Models. In detail, we first train neural language models with a novel dependency modeling objective. We then formulate the next-token probability by mixing the previous dependency modeling probability distributions with self-attention.
  arXiv  Detail & Related papers  (2022-03-19T06:28:30Z)
• Model-based analysis of brain activity reveals the hierarchy of language in 305 subjects [82.81964713263483]
  A popular approach to decompose the neural bases of language consists in correlating, across individuals, the brain responses to different stimuli. Here, we show that a model-based approach can reach equivalent results within subjects exposed to natural stimuli.
  arXiv  Detail & Related papers  (2021-10-12T15:30:21Z)
• Parameter Space Factorization for Zero-Shot Learning across Tasks and Languages [112.65994041398481]
  We propose a Bayesian generative model for the space of neural parameters. We infer the posteriors over such latent variables based on data from seen task-language combinations. Our model yields comparable or better results than state-of-the-art, zero-shot cross-lingual transfer methods.
  arXiv  Detail & Related papers  (2020-01-30T16:58:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.