Inpainting the Neural Picture: Inferring Unrecorded Brain Area Dynamics from Multi-Animal Datasets

• URL: http://arxiv.org/abs/2510.11924v1
• Date: Mon, 13 Oct 2025 20:45:06 GMT
• Title: Inpainting the Neural Picture: Inferring Unrecorded Brain Area Dynamics from Multi-Animal Datasets
• Authors: Ji Xia, Yizi Zhang, Shuqi Wang, Genevera I. Allen, Liam Paninski, Cole Lincoln Hurwitz, Kenneth D. Miller,
• Abstract summary: We introduce NeuroPaint, a masked autoencoding approach for inferring dynamics of unrecorded brain areas.<n>By training across animals with overlapping subsets of recorded areas, NeuroPaint learns to reconstruct activity in missing areas based on shared structure across individuals.<n>Our results demonstrate that models trained across animals with partial observations can successfully in-paint the dynamics of unrecorded areas.
• Score: 10.096134908121904
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Characterizing interactions between brain areas is a fundamental goal of systems neuroscience. While such analyses are possible when areas are recorded simultaneously, it is rare to observe all combinations of areas of interest within a single animal or recording session. How can we leverage multi-animal datasets to better understand multi-area interactions? Building on recent progress in large-scale, multi-animal models, we introduce NeuroPaint, a masked autoencoding approach for inferring the dynamics of unrecorded brain areas. By training across animals with overlapping subsets of recorded areas, NeuroPaint learns to reconstruct activity in missing areas based on shared structure across individuals. We train and evaluate our approach on synthetic data and two multi-animal, multi-area Neuropixels datasets. Our results demonstrate that models trained across animals with partial observations can successfully in-paint the dynamics of unrecorded areas, enabling multi-area analyses that transcend the limitations of any single experiment.

Related papers

• Neural Encoding and Decoding at Scale [42.33285735011587]
  We introduce a multimodal, multi-task model that enables simultaneous Neural and Decoding at Scale (NEDS)<n>Central to our approach is a novel multi-task-masking strategy, which alternates between neural, behavioral, within-modality, and cross-modality masking.<n>NEDS achieves state-of-the-art performance for both encoding and decoding when pretrained on multi-animal data and then fine-tuned on new animals.
  arXiv  Detail & Related papers  (2025-04-11T02:06:20Z)
• BrainMAP: Learning Multiple Activation Pathways in Brain Networks [77.15180533984947]
  We introduce a novel framework BrainMAP to learn Multiple Activation Pathways in Brain networks.<n>Our framework enables explanatory analyses of crucial brain regions involved in tasks.
  arXiv  Detail & Related papers  (2024-12-23T09:13:35Z)
• Towards a "universal translator" for neural dynamics at single-cell, single-spike resolution [10.49121904052395]
  We build towards a first foundation model for neural spiking data that can solve a diverse set of tasks across multiple brain areas. Prediction tasks include single-neuron and region-level activity prediction, forward prediction, and behavior decoding.
  arXiv  Detail & Related papers  (2024-07-19T21:05:28Z)
• A Unified, Scalable Framework for Neural Population Decoding [12.052847252465826]
  We introduce a training framework and architecture designed to model the population dynamics of neural activity. We construct a large-scale multi-session model trained on large datasets from seven nonhuman primates.
  arXiv  Detail & Related papers  (2023-10-24T17:58:26Z)
• Seeing the forest and the tree: Building representations of both individual and collective dynamics with transformers [6.543007700542191]
  We present a novel transformer architecture for learning from time-varying data. We show that our model can be applied to successfully recover complex interactions and dynamics in many-body systems. Our results show that it is possible to learn from neurons in one animal's brain and transfer the model on neurons in a different animal's brain, with interpretable neuron correspondence across sets and animals.
  arXiv  Detail & Related papers  (2022-06-10T07:14:57Z)
• Overcoming the Domain Gap in Neural Action Representations [60.47807856873544]
  3D pose data can now be reliably extracted from multi-view video sequences without manual intervention. We propose to use it to guide the encoding of neural action representations together with a set of neural and behavioral augmentations. To reduce the domain gap, during training, we swap neural and behavioral data across animals that seem to be performing similar actions.
  arXiv  Detail & Related papers  (2021-12-02T12:45:46Z)
• Overcoming the Domain Gap in Contrastive Learning of Neural Action Representations [60.47807856873544]
  A fundamental goal in neuroscience is to understand the relationship between neural activity and behavior. We generated a new multimodal dataset consisting of the spontaneous behaviors generated by fruit flies. This dataset and our new set of augmentations promise to accelerate the application of self-supervised learning methods in neuroscience.
  arXiv  Detail & Related papers  (2021-11-29T15:27:51Z)
• Cetacean Translation Initiative: a roadmap to deciphering the communication of sperm whales [97.41394631426678]
  Recent research showed the promise of machine learning tools for analyzing acoustic communication in nonhuman species. We outline the key elements required for the collection and processing of massive bioacoustic data of sperm whales. The technological capabilities developed are likely to yield cross-applications and advancements in broader communities investigating non-human communication and animal behavioral research.
  arXiv  Detail & Related papers  (2021-04-17T18:39:22Z)
• Muti-view Mouse Social Behaviour Recognition with Deep Graphical Model [124.26611454540813]
  Social behaviour analysis of mice is an invaluable tool to assess therapeutic efficacy of neurodegenerative diseases. Because of the potential to create rich descriptions of mouse social behaviors, the use of multi-view video recordings for rodent observations is increasingly receiving much attention. We propose a novel multiview latent-attention and dynamic discriminative model that jointly learns view-specific and view-shared sub-structures.
  arXiv  Detail & Related papers  (2020-11-04T18:09:58Z)
• Modeling Shared Responses in Neuroimaging Studies through MultiView ICA [94.31804763196116]
  Group studies involving large cohorts of subjects are important to draw general conclusions about brain functional organization. We propose a novel MultiView Independent Component Analysis model for group studies, where data from each subject are modeled as a linear combination of shared independent sources plus noise. We demonstrate the usefulness of our approach first on fMRI data, where our model demonstrates improved sensitivity in identifying common sources among subjects.
  arXiv  Detail & Related papers  (2020-06-11T17:29:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.