Topological Signatures of Grokking

• URL: http://arxiv.org/abs/2605.06352v1
• Date: Thu, 07 May 2026 14:33:22 GMT
• Title: Topological Signatures of Grokking
• Abstract summary: We study the grokking phenomenon through the lens of topology.<n> Persistent homology provides a unified geometric and topological characterization of representation learning.<n>Our results suggest that persistent homology offers a principled and interpretable framework for analyzing how neural networks latent structure during training.
• Score: 5.246889031503699
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We study the grokking phenomenon through the lens of topology. Using persistent homology on point clouds derived from the embedding matrices of a range of models trained on modular arithmetic with varying primes, we identify a clear and consistent topological signature of grokking: a sharp increase in both the maximum and total persistence of first homology ($H_1$). Persistence diagrams reveal the emergence of a dominant long-lived topological feature together with increasingly structured secondary features, reflecting the underlying cyclic structure of the task. Compared to existing spectral and geometric diagnostics -- specifically, Fourier analysis and local intrinsic dimension -- persistent homology provides a unified geometric and topological characterization of representation learning, capturing both local and global multi-scale structure. Ablations across data regimes and control settings show that these topological transitions are tied to generalization rather than memorization. Our results suggest that persistent homology offers a principled and interpretable framework for analyzing how neural networks internalize latent structure during training.

Related papers

• Learning to Tessellate: Point Cloud Generation via Recursive Spectral Partitioning [58.130498928480456]
  We present PointRSP, an autoregressive framework that reformulates point cloud generation as a topology-preserving tessellation process.<n>PointRSP achieves state-of-the-art performance in generation quality and diversity, demonstrating strong generalization across complex 3D topologies.
  arXiv  Detail & Related papers  (2026-08-03T16:11:43Z)
• PathAR: Structure-First Autoregressive Synthesis of Multimodal Pathology Images [51.428093790826814]
  We propose pathology Autorgressive modeling (PathAR) to factorize structure and appearance for modality-label-conditioned pathology generation.<n>PathAR stabilizes morphology under heterogeneous modality-specific appearances and enables spatially aligned image-mask pair generation.
  arXiv  Detail & Related papers  (2026-06-01T01:43:48Z)
• Topological Detection of Hopf Bifurcations via Persistent Homology: A Functional Criterion from Time Series [0.0]
  We propose a framework for the detection of Hopf bifurcations directly from time series, based on persistent homology applied to phase space reconstructions via Takens embedding within the framework of Topological Data Analysis.<n>The results support the interpretation of dynamical transitions as topological phase transitions and demonstrate the potential of topological data analysis as a model-free tool for the quantitative analysis of nonlinear time series.
  arXiv  Detail & Related papers  (2026-03-28T20:25:38Z)
• Bridging Structure and Appearance: Topological Features for Robust Self-Supervised Segmentation [8.584363058858935]
  Self-supervised semantic segmentation methods often fail when faced with appearance ambiguities.<n>We argue that this is due to an over-reliance on unstable, appearance-based features such as shadows, glare, and local textures.<n>We propose textbfGASeg, a novel framework that bridges appearance and geometry by leveraging stable topological information.
  arXiv  Detail & Related papers  (2025-12-30T05:34:28Z)
• Persistent Topological Structures and Cohomological Flows as a Mathematical Framework for Brain-Inspired Representation Learning [0.0]
  This paper presents a mathematically rigorous framework for brain-inspired representation learning.<n>It integrates algebraic topology with differential geometry to construct cohomological operators that generalize gradient-based learning.<n> Empirical results demonstrate that the model achieves superior manifold consistency and noise resilience compared to graph neural and manifold-based deep architectures.
  arXiv  Detail & Related papers  (2025-12-09T04:48:50Z)
• GeoGNN: Quantifying and Mitigating Semantic Drift in Text-Attributed Graphs [59.61242815508687]
  Graph neural networks (GNNs) on text--attributed graphs (TAGs) encode node texts using pretrained language models (PLMs) and propagate these embeddings through linear neighborhood aggregation.<n>This work introduces a local PCA-based metric that measures the degree of semantic drift and provides the first quantitative framework to analyze how different aggregation mechanisms affect manifold structure.
  arXiv  Detail & Related papers  (2025-11-12T06:48:43Z)
• Conformable Convolution for Topologically Aware Learning of Complex Anatomical Structures [38.20599800950335]
  We introduce Conformable Convolution, a novel convolutional layer designed to explicitly enforce topological consistency.<n>Topological Posterior Generator (TPG) module identifies key topological features and guides the convolutional layers.<n>We showcase the effectiveness of our framework in the segmentation task, where preserving the interconnectedness of structures is critical.
  arXiv  Detail & Related papers  (2024-12-29T22:41:33Z)
• PhyloGen: Language Model-Enhanced Phylogenetic Inference via Graph Structure Generation [50.80441546742053]
  Phylogenetic trees elucidate evolutionary relationships among species.<n>Traditional Markov Chain Monte Carlo methods face slow convergence and computational burdens.<n>We propose PhyloGen, a novel method leveraging a pre-trained genomic language model.
  arXiv  Detail & Related papers  (2024-12-25T08:33:05Z)
• Relative Representations: Topological and Geometric Perspectives [50.85040046976025]
  Relative representations are an established approach to zero-shot model stitching.<n>We introduce a normalization procedure in the relative transformation, resulting in invariance to non-isotropic rescalings and permutations.<n>Second, we propose to deploy topological densification when fine-tuning relative representations, a topological regularization loss encouraging clustering within classes.
  arXiv  Detail & Related papers  (2024-09-17T08:09:22Z)
• $k$-Means Clustering for Persistent Homology [0.0]
  We prove convergence of the $k$-means clustering algorithm on persistence diagram space. We also establish theoretical properties of the solution to the optimization problem in the Karush--Kuhn--Tucker framework.
  arXiv  Detail & Related papers  (2022-10-18T17:18:51Z)
• Dist2Cycle: A Simplicial Neural Network for Homology Localization [66.15805004725809]
  Simplicial complexes can be viewed as high dimensional generalizations of graphs that explicitly encode multi-way ordered relations. We propose a graph convolutional model for learning functions parametrized by the $k$-homological features of simplicial complexes.
  arXiv  Detail & Related papers  (2021-10-28T14:59:41Z)
• Bayesian Topological Learning for Classifying the Structure of Biological Networks [1.644043499620662]
  Actin cytoskeleton networks generate local topological signatures due to the natural variations in the number, size, and shape of holes of the networks. Persistent homology is a method that explores these topological properties of data and summarizes them as persistence diagrams. We implement a Bayes factor algorithm to classify the actin filament networks and benchmark it against several state-of-the-art classification methods.
  arXiv  Detail & Related papers  (2020-09-24T22:43:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.