Network-Aware Bilinear Tokenization for Brain Functional Connectivity Representation Learning

• URL: http://arxiv.org/abs/2605.14048v3
• Date: Tue, 19 May 2026 14:47:48 GMT
• Title: Network-Aware Bilinear Tokenization for Brain Functional Connectivity Representation Learning
• Abstract summary: We introduce NERVE, a self-supervised learning framework that redefines brain functional connectivity tokenization.<n>Unlike image-based MAEs, FC patches defined by network pairs are heterogeneous in size and correspond to distinct functional roles.<n>NERVE embeds FC patches through a novel structured bilinear factorization.
• Score: 6.232102297445596
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Masked autoencoders (MAEs) have recently shown promise for self-supervised representation learning of resting-state brain functional connectivity (FC). However, a fundamental question remains unresolved: how should FC matrices be tokenized to align with the intrinsic modular organization of large-scale brain networks? Existing approaches typically adopt region-centric or graph-based schemes that treat FC as structurally homogeneous elements and overlook the large-scale network brain organization. We introduce NERVE (Network-Aware Representations of Brain Functional Connectivity via Bilinear Tokenization), a self-supervised learning framework that redefines FC tokenization by partitioning FC matrices into patches of intra- and inter-network connectivity blocks. Unlike image-based MAE, where fixed-size patches share a common tokenizer, FC patches defined by network pairs are heterogeneous in size and correspond to distinct functional roles. To resolve this problem, NERVE embeds FC patches through a novel structured bilinear factorization. This formulation preserves network identity and reduces parameter complexity from quadratic to linear scaling in the number of networks. We evaluate NERVE across three large-scale developmental cohorts (ABCD, PNC, and CCNP) for behavior and psychopathology prediction. Compared to structurally agnostic MAE variants and graph-based self-supervised baselines, the proposed network-aware formulation yields more stable and transferable representations, particularly in cross-cohort evaluation. Ablation studies confirm that the proposed bilinear network embedding and anatomically grounded parcellation are critical for performance. These findings highlight the importance of incorporating domain-specific structural priors into self-supervised learning for functional connectomics. Code is available at: https://github.com/leomlck/NERVE.

Related papers

• Rare Events, Real Signals: Functional Ensembles as Units of Computation in Deep Spiking Networks [0.0]
  We investigate how internal representations emerge across hierarchical processing systems by introducing a neuroscience-inspired framework for analyzing deep spiking neural networks (SNN)<n>We form the first-order functionally-connected (1FC) group of a neuron based on its statistically significant pairwise correlations with neurons from the previous layer of a trained SNN architecture.<n>Our analysis shows that several principles of functional connectivity previously observed in biological cortex are preserved in spiking ResNet architectures.
  arXiv  Detail & Related papers  (2026-05-21T15:44:32Z)
• Learning Structural-Functional Brain Representations through Multi-Scale Adaptive Graph Attention for Cognitive Insight [15.13186621391124]
  We introduce Multi-scale Adaptive Graph Network (MAGNet), a graph neural network framework that adaptively learns structure-function interactions.<n>A hybrid graph integrates direct and indirect pathways, while local-global attention refines connectivity importance and a joint loss enforces cross-modal coherence.<n>On the ABCD dataset, MAGNet outperformed relevant baselines, demonstrating effective multimodal integration for advancing our understanding of cognitive function.
  arXiv  Detail & Related papers  (2026-03-31T16:32:15Z)
• Toward a Multi-View Brain Network Foundation Model: Cross-View Consistency Learning Across Arbitrary Atlases [62.33465338932216]
  MV-BrainFM is a multi-view brain network foundation model designed to learn generalizable and scalable representations from brain networks constructed with arbitrary atlases.<n>Experiments on more than 20K subjects from 17 fMRI datasets show that MV-BrainFM consistently outperforms 14 existing brain network foundation models.
  arXiv  Detail & Related papers  (2026-03-20T11:55:00Z)
• Learning the Hierarchical Organization in Brain Network for Brain Disorder Diagnosis [10.444771588267436]
  We propose the Brain Hierarchical Organization Learning (BrainHO) to learn inherently hierarchical brain network dependencies.<n>Specifically, we design a hierarchical attention mechanism that allows the model to aggregate nodes into a hierarchical organization.<n>Experiments on the publicly available ABIDE and REST-meta-MDD datasets demonstrate that BrainHO not only achieves state-of-the-art classification performance but also uncovers interpretable, clinically significant biomarkers.
  arXiv  Detail & Related papers  (2026-03-10T12:49:45Z)
• Interpretable Cross-Network Attention for Resting-State fMRI Representation Learning [5.79619045482301]
  BrainInterNet is a network-aware self-supervised framework based on masked reconstruction with cross-attention.<n>We train BrainInterNet on multi-cohort fMRI data and evaluate on the Alzheimer's Disease Neuroimaging Initiative dataset.<n>Our method reveals systematic alterations in the brain's network interactions under Alzheimer's disease.
  arXiv  Detail & Related papers  (2026-02-28T19:15:52Z)
• Prototype Learning Guided Hybrid Network for Breast Tumor Segmentation in DCE-MRI [58.809276442508256]
  We propose a hybrid network via the combination of convolution neural network (CNN) and transformer layers. The experimental results on private and public DCE-MRI datasets demonstrate that the proposed hybrid network superior performance than the state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-08-11T15:46:00Z)
• Semantic Loss Functions for Neuro-Symbolic Structured Prediction [74.18322585177832]
  We discuss the semantic loss, which injects knowledge about such structure, defined symbolically, into training. It is agnostic to the arrangement of the symbols, and depends only on the semantics expressed thereby. It can be combined with both discriminative and generative neural models.
  arXiv  Detail & Related papers  (2024-05-12T22:18:25Z)
• Skin Lesion Segmentation Improved by Transformer-based Networks with Inter-scale Dependency Modeling [0.0]
  Melanoma is a dangerous type of skin cancer resulting from abnormal skin cell growth. The symmetrical U-Net model's reliance on convolutional operations hinders its ability to capture long-range dependencies. Several Transformer-based U-Net topologies have recently been created to overcome this limitation.
  arXiv  Detail & Related papers  (2023-10-20T15:53:51Z)
• Local Kernel Renormalization as a mechanism for feature learning in overparametrized Convolutional Neural Networks [0.0]
  Empirical evidence shows that fully-connected neural networks in the infinite-width limit eventually outperform their finite-width counterparts. State-of-the-art architectures with convolutional layers achieve optimal performances in the finite-width regime. We show that the generalization performance of a finite-width FC network can be obtained by an infinite-width network, with a suitable choice of the Gaussian priors.
  arXiv  Detail & Related papers  (2023-07-21T17:22:04Z)
• Learning Interpretable Models for Coupled Networks Under Domain Constraints [8.308385006727702]
  We investigate the idea of coupled networks by focusing on interactions between structural edges and functional edges of brain networks. We propose a novel formulation to place hard network constraints on the noise term while estimating interactions. We validate our method on multishell diffusion and task-evoked fMRI datasets from the Human Connectome Project.
  arXiv  Detail & Related papers  (2021-04-19T06:23:31Z)
• Multiple-view clustering for identifying subject clusters and brain sub-networks using functional connectivity matrices without vectorization [0.0]
  We propose a novel multiple-view clustering method based on Wishart mixture models. The uniqueness of this method is that the multiple-view clustering of subjects is based on particular networks of nodes.
  arXiv  Detail & Related papers  (2020-10-20T00:51:45Z)
• Dual-constrained Deep Semi-Supervised Coupled Factorization Network with Enriched Prior [80.5637175255349]
  We propose a new enriched prior based Dual-constrained Deep Semi-Supervised Coupled Factorization Network, called DS2CF-Net. To ex-tract hidden deep features, DS2CF-Net is modeled as a deep-structure and geometrical structure-constrained neural network. Our network can obtain state-of-the-art performance for representation learning and clustering.
  arXiv  Detail & Related papers  (2020-09-08T13:10:21Z)
• Learning Connectivity of Neural Networks from a Topological Perspective [80.35103711638548]
  We propose a topological perspective to represent a network into a complete graph for analysis. By assigning learnable parameters to the edges which reflect the magnitude of connections, the learning process can be performed in a differentiable manner. This learning process is compatible with existing networks and owns adaptability to larger search spaces and different tasks.
  arXiv  Detail & Related papers  (2020-08-19T04:53:31Z)
• Context-Aware Refinement Network Incorporating Structural Connectivity Prior for Brain Midline Delineation [50.868845400939314]
  We propose a context-aware refinement network (CAR-Net) to refine and integrate the feature pyramid representation generated by the UNet. For keeping the structural connectivity of the brain midline, we introduce a novel connectivity regular loss. The proposed method requires fewer parameters and outperforms three state-of-the-art methods in terms of four evaluation metrics.
  arXiv  Detail & Related papers  (2020-07-10T14:01:20Z)
• MDP Homomorphic Networks: Group Symmetries in Reinforcement Learning [90.20563679417567]
  This paper introduces MDP homomorphic networks for deep reinforcement learning. MDP homomorphic networks are neural networks that are equivariant under symmetries in the joint state-action space of an MDP. We show that such networks converge faster than unstructured networks on CartPole, a grid world and Pong.
  arXiv  Detail & Related papers  (2020-06-30T15:38:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.