Local Intrinsic Dimension of Representations Predicts Alignment and Generalization in AI Models and Human Brain

• URL: http://arxiv.org/abs/2601.22722v1
• Date: Fri, 30 Jan 2026 08:54:59 GMT
• Title: Local Intrinsic Dimension of Representations Predicts Alignment and Generalization in AI Models and Human Brain
• Authors: Junjie Yu, Wenxiao Ma, Chen Wei, Jianyu Zhang, Haotian Deng, Zihan Deng, Quanying Liu,
• Abstract summary: Recent work has found that neural networks with stronger generalization tend to exhibit higher representational alignment with one another.<n>We show that models with stronger generalization also align more strongly with human neural activity.<n>These relationships can be explained by a single geometric property of learned representations: the local intrinsic dimension of embeddings.
• Score: 14.072972213206524
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent work has found that neural networks with stronger generalization tend to exhibit higher representational alignment with one another across architectures and training paradigms. In this work, we show that models with stronger generalization also align more strongly with human neural activity. Moreover, generalization performance, model--model alignment, and model--brain alignment are all significantly correlated with each other. We further show that these relationships can be explained by a single geometric property of learned representations: the local intrinsic dimension of embeddings. Lower local dimension is consistently associated with stronger model--model alignment, stronger model--brain alignment, and better generalization, whereas global dimension measures fail to capture these effects. Finally, we find that increasing model capacity and training data scale systematically reduces local intrinsic dimension, providing a geometric account of the benefits of scaling. Together, our results identify local intrinsic dimension as a unifying descriptor of representational convergence in artificial and biological systems.

Related papers

• Geometric Priors for Generalizable World Models via Vector Symbolic Architecture [9.216794073296679]
  Key challenge in artificial intelligence is understanding how neural systems learn representations that capture the underlying dynamics of the world.<n>We address these issues with a generalizable world model grounded in Vector Symbolic Architecture (VSA) principles as geometric priors.<n>We show how training to have latent group structure yields generalizable, data-efficient, and interpretable world models.
  arXiv  Detail & Related papers  (2026-02-25T00:41:42Z)
• Universally Converging Representations of Matter Across Scientific Foundation Models [5.309886698585678]
  We show that representations learned by nearly sixty scientific models are highly aligned across a wide range of chemical systems.<n>On inputs similar to those seen during training, high-performing models align closely and weak models diverge into local sub-optima in representation space.<n>Our findings establish representational alignment as a quantitative benchmark for foundation-level generality in scientific models.
  arXiv  Detail & Related papers  (2025-12-03T12:47:06Z)
• Integrated representational signatures strengthen specificity in brains and models [8.045700364123645]
  Similarity Network Fusion (SNF) is a framework originally developed for multi-omics data integration.<n>SNF produces substantially sharper regional and model family-level separation than any single metric.<n>Clustering cortical regions using SNF-derived similarity scores reveals a clearer hierarchical organization.
  arXiv  Detail & Related papers  (2025-10-21T04:37:27Z)
• Random Matrix Theory for Deep Learning: Beyond Eigenvalues of Linear Models [51.85815025140659]
  Modern Machine Learning (ML) and Deep Neural Networks (DNNs) often operate on high-dimensional data.<n>In particular, the proportional regime where the data dimension, sample size, and number of model parameters are all large gives rise to novel and sometimes counterintuitive behaviors.<n>This paper extends traditional Random Matrix Theory (RMT) beyond eigenvalue-based analysis of linear models to address the challenges posed by nonlinear ML models.
  arXiv  Detail & Related papers  (2025-06-16T06:54:08Z)
• High-Fidelity Scientific Simulation Surrogates via Adaptive Implicit Neural Representations [51.90920900332569]
  Implicit neural representations (INRs) offer a compact and continuous framework for modeling spatially structured data.<n>Recent approaches address this by introducing additional features along rigid geometric structures.<n>We propose a simple yet effective alternative: Feature-Adaptive INR (FA-INR)
  arXiv  Detail & Related papers  (2025-06-07T16:45:17Z)
• Connecting Neural Models Latent Geometries with Relative Geodesic Representations [21.71782603770616]
  We show that when a latent structure is shared between distinct latent spaces, relative distances between representations can be preserved, up to distortions.<n>We assume that distinct neural models parametrize approximately the same underlying manifold, and introduce a representation based on the pullback metric.<n>We validate our method on model stitching and retrieval tasks, covering autoencoders and vision foundation discriminative models.
  arXiv  Detail & Related papers  (2025-06-02T12:34:55Z)
• Learning Covariance-Based Multi-Scale Representation of Neuroimaging Measures for Alzheimer Classification [5.427921447614832]
  We present a framework capable of deriving an efficient high-dimensional space with reasonable increase in model size.<n>Experiments on neuroimaging measures from Alzheimer's Disease Neuroimaging Initiative (ADNI) study show that our model performs better.<n>The trained model is made interpretable using gradient information over the multi-scale transform to delineate personalized AD-specific regions in the brain.
  arXiv  Detail & Related papers  (2025-03-03T06:55:35Z)
• Scaling Laws Do Not Scale [54.72120385955072]
  Recent work has argued that as the size of a dataset increases, the performance of a model trained on that dataset will increase. We argue that this scaling law relationship depends on metrics used to measure performance that may not correspond with how different groups of people perceive the quality of models' output. Different communities may also have values in tension with each other, leading to difficult, potentially irreconcilable choices about metrics used for model evaluations.
  arXiv  Detail & Related papers  (2023-07-05T15:32:21Z)
• Towards Understanding and Mitigating Dimensional Collapse in Heterogeneous Federated Learning [112.69497636932955]
  Federated learning aims to train models across different clients without the sharing of data for privacy considerations. We study how data heterogeneity affects the representations of the globally aggregated models. We propose sc FedDecorr, a novel method that can effectively mitigate dimensional collapse in federated learning.
  arXiv  Detail & Related papers  (2022-10-01T09:04:17Z)
• Generalized Shape Metrics on Neural Representations [26.78835065137714]
  We provide a family of metric spaces that quantify representational dissimilarity. We modify existing representational similarity measures based on canonical correlation analysis to satisfy the triangle inequality. We identify relationships between neural representations that are interpretable in terms of anatomical features and model performance.
  arXiv  Detail & Related papers  (2021-10-27T19:48:55Z)
• Post-mortem on a deep learning contest: a Simpson's paradox and the complementary roles of scale metrics versus shape metrics [61.49826776409194]
  We analyze a corpus of models made publicly-available for a contest to predict the generalization accuracy of neural network (NN) models. We identify what amounts to a Simpson's paradox: where "scale" metrics perform well overall but perform poorly on sub partitions of the data. We present two novel shape metrics, one data-independent, and the other data-dependent, which can predict trends in the test accuracy of a series of NNs.
  arXiv  Detail & Related papers  (2021-06-01T19:19:49Z)
• Hyperbolic Neural Networks++ [66.16106727715061]
  We generalize the fundamental components of neural networks in a single hyperbolic geometry model, namely, the Poincar'e ball model. Experiments show the superior parameter efficiency of our methods compared to conventional hyperbolic components, and stability and outperformance over their Euclidean counterparts.
  arXiv  Detail & Related papers  (2020-06-15T08:23:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.