CAPT: A Multi-task Continuous Autoregressive Transformer enabling Cross-dataset and Cross-species Transfer for Calcium Population Dynamics

• URL: http://arxiv.org/abs/2607.23258v1
• Date: Sat, 25 Jul 2026 15:53:55 GMT
• Title: CAPT: A Multi-task Continuous Autoregressive Transformer enabling Cross-dataset and Cross-species Transfer for Calcium Population Dynamics
• Abstract summary: We present textbfCAPT, a textbfContinuous textbfAutoregressive textbfPopulation textbfTransformer for calcium population dynamics.<n>CAPT models continuous calcium traces directly through a continuous patch tokenization strategy and is trained autoregressively.<n>Across neural population forecasting and behavior decoding tasks, CAPT consistently outperforms specialized and general-purpose baselines.
• Score: 9.819202688456096
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large-scale calcium imaging has created an opportunity to build foundation-style models for neural population dynamics, but a central question remains unresolved: \textbf{whether a model pretrained on one collection of recordings can generalize to new datasets, experimental paradigms, and even species.} Existing approaches are often designed for specific tasks and evaluated on a single dataset, making it unclear whether their learned representations are reusable for new calcium trace datasets. To tackle this gap, we present \textbf{CAPT}, a \textbf{C}ontinuous \textbf{A}utoregressive \textbf{P}opulation \textbf{T}ransformer for calcium population dynamics. CAPT models continuous calcium traces directly through a continuous patch tokenization strategy and is trained autoregressively, enabling end-to-end pretraining and adaptation to diverse downstream tasks. We first pretrain CAPT on a large-scale mouse calcium imaging dataset and evaluate its transferability across independent mouse, larval zebrafish, and \textit{C. elegans} datasets collected by different laboratories. In these transfer settings, the pretrained backbone is frozen and only adaptation modules are updated. Across neural population forecasting and behavior decoding tasks, CAPT consistently outperforms specialized and general-purpose baselines. Alongside predictive performance, multimodal analyses using NeuroPAL annotations in \textit{C. elegans} datasets show that CAPT embeddings form a shared functional space across datasets and capture anatomical cell-identity-related structure. These results suggest that the continuous autoregressive modeling opens up possibilities for a simple route towards general-purpose neural foundation models for calcium imaging, which can generalize across datasets, experimental paradigms, and species.

Related papers

• Miles: Metric Learning with Expandable Subspace for Pre-Trained Model-Based Class-Incremental Learning [57.237835833636154]
  We propose MetrIc Learning with Expandable Subspace (Miles) to harness the prior information within pre-trained knowledge.<n>Miles achieves state-of-the-art performance in various CIL settings.
  arXiv  Detail & Related papers  (2026-07-20T06:22:02Z)
• Cross-Subject Modeling for Widefield Calcium Imaging via Atlas-Aligned Spatiotemporal Tokenization [0.20415910628419062]
  Large-scale, multi-subject widefield calcium imaging provides unprecedented access to brain-wide cortical dynamics.<n>We introduce WiCAT, a multi-subject model that leverages self-trained pretraining to both outperform single-session models and generalizations.<n>Across multiple widefield datasets, WiCAT supports lightweight downstream decoding, across subjects, tasks, and decoding, and outperforms baseline models.
  arXiv  Detail & Related papers  (2026-07-05T01:10:52Z)
• Self-Supervised Foundation Model for Calcium-imaging Population Dynamics [8.988419599192138]
  We propose textbfCalM, a self-supervised neural foundation model trained solely on calcium traces.<n>We evaluate CalM on a large-scale, multi-animal, multi-session dataset.
  arXiv  Detail & Related papers  (2026-04-03T13:46:41Z)
• Decoding Dynamic Visual Experience from Calcium Imaging via Cell-Pattern-Aware SSL [6.099145932421348]
  We present a novel approach to self-supervised pretraining, POYO-SSL, that exploits the heterogeneity of neural data to improve pre-training and achieve benefits of scale.<n>Specifically, in POYO-SSL we pretrain only on predictable (statistically regular) neurons-identified on the pretraining split via simple higher-order statistics (skewness and kurtosis)-then we fine-tune on the unpredictable population for downstream tasks.<n>On the Allen Brain Observatory dataset, this strategy yields approximately 12-13% relative gains over from-scratch training and exhibits smooth, monotonic scaling with model size.
  arXiv  Detail & Related papers  (2025-10-21T10:57:52Z)
• POCO: Scalable Neural Forecasting through Population Conditioning [6.135391899876823]
  POCO is a unified forecasting model that captures both neuron-specific and brain-wide dynamics.<n>After pre-training, POCO rapidly adapts to new recordings with minimal fine-tuning.<n>Our comprehensive analysis reveals several key factors influencing performance, including context length, session diversity, and preprocessing.
  arXiv  Detail & Related papers  (2025-06-17T20:15:04Z)
• Meta Flow Matching: Integrating Vector Fields on the Wasserstein Manifold [83.18058549195855]
  We argue that multiple processes in natural sciences have to be represented as vector fields on the Wasserstein manifold of probability densities.<n>In particular, this is crucial for personalized medicine where the development of diseases and their respective treatment response depend on the microenvironment of cells specific to each patient.<n>We propose Meta Flow Matching (MFM), a practical approach to integrate along these vector fields on the Wasserstein manifold by amortizing the flow model over the initial populations.
  arXiv  Detail & Related papers  (2024-08-26T20:05:31Z)
• TokenUnify: Scaling Up Autoregressive Pretraining for Neuron Segmentation [65.65530016765615]
  We propose a hierarchical predictive coding framework that captures multi-scale dependencies through three complementary learning objectives.<n> TokenUnify integrates random token prediction, next-token prediction, and next-all token prediction to create a comprehensive representational space.<n>We also introduce a large-scale EM dataset with 1.2 billion annotated voxels, offering ideal long-sequence visual data with spatial continuity.
  arXiv  Detail & Related papers  (2024-05-27T05:45:51Z)
• Skeleton2vec: A Self-supervised Learning Framework with Contextualized Target Representations for Skeleton Sequence [56.092059713922744]
  We show that using high-level contextualized features as prediction targets can achieve superior performance. Specifically, we propose Skeleton2vec, a simple and efficient self-supervised 3D action representation learning framework. Our proposed Skeleton2vec outperforms previous methods and achieves state-of-the-art results.
  arXiv  Detail & Related papers  (2024-01-01T12:08:35Z)
• Meta Transfer of Self-Supervised Knowledge: Foundation Model in Action for Post-Traumatic Epilepsy Prediction [0.6291443816903801]
  We introduce a novel training strategy for our foundation model. We demonstrate that the proposed strategy significantly improves task performance on small-scale clinical datasets. Results further demonstrated the enhanced generalizability of our foundation model.
  arXiv  Detail & Related papers  (2023-12-21T07:42:49Z)
• Kalman Filter for Online Classification of Non-Stationary Data [101.26838049872651]
  In Online Continual Learning (OCL) a learning system receives a stream of data and sequentially performs prediction and training steps. We introduce a probabilistic Bayesian online learning model by using a neural representation and a state space model over the linear predictor weights. In experiments in multi-class classification we demonstrate the predictive ability of the model and its flexibility to capture non-stationarity.
  arXiv  Detail & Related papers  (2023-06-14T11:41:42Z)
• Learning from Temporal Spatial Cubism for Cross-Dataset Skeleton-based Action Recognition [88.34182299496074]
  Action labels are only available on a source dataset, but unavailable on a target dataset in the training stage. We utilize a self-supervision scheme to reduce the domain shift between two skeleton-based action datasets. By segmenting and permuting temporal segments or human body parts, we design two self-supervised learning classification tasks.
  arXiv  Detail & Related papers  (2022-07-17T07:05:39Z)
• PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning [109.84770951839289]
  We present PredRNN, a new recurrent network for learning visual dynamics from historical context. We show that our approach obtains highly competitive results on three standard datasets.
  arXiv  Detail & Related papers  (2021-03-17T08:28:30Z)
• Towards an Automatic Analysis of CHO-K1 Suspension Growth in Microfluidic Single-cell Cultivation [63.94623495501023]
  We propose a novel Machine Learning architecture, which allows us to infuse a neural deep network with human-powered abstraction on the level of data. Specifically, we train a generative model simultaneously on natural and synthetic data, so that it learns a shared representation, from which a target variable, such as the cell count, can be reliably estimated.
  arXiv  Detail & Related papers  (2020-10-20T08:36:51Z)
• Convolutional Tensor-Train LSTM for Spatio-temporal Learning [116.24172387469994]
  We propose a higher-order LSTM model that can efficiently learn long-term correlations in the video sequence. This is accomplished through a novel tensor train module that performs prediction by combining convolutional features across time. Our results achieve state-of-the-art performance-art in a wide range of applications and datasets.
  arXiv  Detail & Related papers  (2020-02-21T05:00:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.