Multi-task Collaborative Pre-training and Individual-adaptive-tokens
Fine-tuning: A Unified Framework for Brain Representation Learning
- URL: http://arxiv.org/abs/2306.11378v1
- Date: Tue, 20 Jun 2023 08:38:17 GMT
- Title: Multi-task Collaborative Pre-training and Individual-adaptive-tokens
Fine-tuning: A Unified Framework for Brain Representation Learning
- Authors: Ning Jiang, Gongshu Wang, and Tianyi Yan
- Abstract summary: We propose a unified framework that combines Collaborative pre-training and Individual--Tokens fine-tuning.
The proposed MCIAT achieves state-of-the-art diagnosis performance on the ADHD-200 dataset.
- Score: 3.1453938549636185
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Structural magnetic resonance imaging (sMRI) provides accurate estimates of
the brain's structural organization and learning invariant brain
representations from sMRI is an enduring issue in neuroscience. Previous deep
representation learning models ignore the fact that the brain, as the core of
human cognitive activity, is distinct from other organs whose primary attribute
is anatomy. Therefore, capturing the semantic structure that dominates
interindividual cognitive variability is key to accurately representing the
brain. Given that this high-level semantic information is subtle, distributed,
and interdependently latent in the brain structure, sMRI-based models need to
capture fine-grained details and understand how they relate to the overall
global structure. However, existing models are optimized by simple objectives,
making features collapse into homogeneity and worsening simultaneous
representation of fine-grained information and holistic semantics, causing a
lack of biological plausibility and interpretation of cognition. Here, we
propose MCIAT, a unified framework that combines Multi-task Collaborative
pre-training and Individual-Adaptive-Tokens fine-tuning. Specifically, we first
synthesize restorative learning, age prediction auxiliary learning and
adversarial learning as a joint proxy task for deep semantic representation
learning. Then, a mutual-attention-based token selection method is proposed to
highlight discriminative features. The proposed MCIAT achieves state-of-the-art
diagnosis performance on the ADHD-200 dataset compared with several sMRI-based
approaches and shows superior generalization on the MCIC and OASIS datasets.
Moreover, we studied 12 behavioral tasks and found significant associations
between cognitive functions and MCIAT-established representations, which
verifies the interpretability of our proposed framework.
Related papers
- BrainMAE: A Region-aware Self-supervised Learning Framework for Brain Signals [11.030708270737964]
We propose Brain Masked Auto-Encoder (BrainMAE) for learning representations directly from fMRI time-series data.
BrainMAE consistently outperforms established baseline methods by significant margins in four distinct downstream tasks.
arXiv Detail & Related papers (2024-06-24T19:16:24Z) - Interpretable Spatio-Temporal Embedding for Brain Structural-Effective Network with Ordinary Differential Equation [56.34634121544929]
In this study, we first construct the brain-effective network via the dynamic causal model.
We then introduce an interpretable graph learning framework termed Spatio-Temporal Embedding ODE (STE-ODE)
This framework incorporates specifically designed directed node embedding layers, aiming at capturing the dynamic interplay between structural and effective networks.
arXiv Detail & Related papers (2024-05-21T20:37:07Z) - MindBridge: A Cross-Subject Brain Decoding Framework [60.58552697067837]
Brain decoding aims to reconstruct stimuli from acquired brain signals.
Currently, brain decoding is confined to a per-subject-per-model paradigm.
We present MindBridge, that achieves cross-subject brain decoding by employing only one model.
arXiv Detail & Related papers (2024-04-11T15:46:42Z) - Psychometry: An Omnifit Model for Image Reconstruction from Human Brain Activity [60.983327742457995]
Reconstructing the viewed images from human brain activity bridges human and computer vision through the Brain-Computer Interface.
We devise Psychometry, an omnifit model for reconstructing images from functional Magnetic Resonance Imaging (fMRI) obtained from different subjects.
arXiv Detail & Related papers (2024-03-29T07:16:34Z) - Brain-ID: Learning Contrast-agnostic Anatomical Representations for
Brain Imaging [11.06907516321673]
We introduce Brain-ID, an anatomical representation learning model for brain imaging.
With the proposed "mild-to-severe" intrasubject generation, Brain-ID is robust to the subject-specific brain anatomy.
We present new metrics to validate the intra- and inter-subject robustness, and evaluate their performance on four downstream applications.
arXiv Detail & Related papers (2023-11-28T16:16:10Z) - UniBrain: Universal Brain MRI Diagnosis with Hierarchical
Knowledge-enhanced Pre-training [66.16134293168535]
We propose a hierarchical knowledge-enhanced pre-training framework for the universal brain MRI diagnosis, termed as UniBrain.
Specifically, UniBrain leverages a large-scale dataset of 24,770 imaging-report pairs from routine diagnostics.
arXiv Detail & Related papers (2023-09-13T09:22:49Z) - Brain Structure-Function Fusing Representation Learning using
Adversarial Decomposed-VAE for Analyzing MCI [17.757114703434027]
A novel brain structure-function fusing-representation learning (BSFL) model is proposed to learn fused representation from fMRI imaging.
The proposed model achieves better performance than other competitive methods in predicting and analyzing mild cognitive impairment (MCI)
The model could be a potential tool for reconstructing unified brain networks and predicting abnormal connections during the degenerative processes in MCI.
arXiv Detail & Related papers (2023-05-23T11:19:02Z) - Language Knowledge-Assisted Representation Learning for Skeleton-Based
Action Recognition [71.35205097460124]
How humans understand and recognize the actions of others is a complex neuroscientific problem.
LA-GCN proposes a graph convolution network using large-scale language models (LLM) knowledge assistance.
arXiv Detail & Related papers (2023-05-21T08:29:16Z) - Functional2Structural: Cross-Modality Brain Networks Representation
Learning [55.24969686433101]
Graph mining on brain networks may facilitate the discovery of novel biomarkers for clinical phenotypes and neurodegenerative diseases.
We propose a novel graph learning framework, known as Deep Signed Brain Networks (DSBN), with a signed graph encoder.
We validate our framework on clinical phenotype and neurodegenerative disease prediction tasks using two independent, publicly available datasets.
arXiv Detail & Related papers (2022-05-06T03:45:36Z) - Mapping individual differences in cortical architecture using multi-view
representation learning [0.0]
We introduce a novel machine learning method which allows combining the activation-and connectivity-based information respectively measured through task-fMRI and resting-state fMRI.
It combines a multi-view deep autoencoder which is designed to fuse the two fMRI modalities into a joint representation space within which a predictive model is trained to guess a scalar score that characterizes the patient.
arXiv Detail & Related papers (2020-04-01T09:01:25Z) - Towards a predictive spatio-temporal representation of brain data [0.2580765958706854]
We show that fMRI datasets are constituted by complex and highly heterogeneous timeseries.
We compare various modelling techniques from deep learning and geometric deep learning to pave the way for future research.
We hope that our methodological advances can ultimately be clinically and computationally relevant by leading to a more nuanced understanding of the brain dynamics in health and disease.
arXiv Detail & Related papers (2020-02-29T18:49:45Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.