Bridging Critical Gaps in Convergent Learning: How Representational Alignment Evolves Across Layers, Training, and Distribution Shifts

• URL: http://arxiv.org/abs/2502.18710v2
• Date: Thu, 29 May 2025 06:26:42 GMT
• Title: Bridging Critical Gaps in Convergent Learning: How Representational Alignment Evolves Across Layers, Training, and Distribution Shifts
• Authors: Chaitanya Kapoor, Sudhanshu Srivastava, Meenakshi Khosla,
• Abstract summary: convergent learning is the degree to which neural systems arrive at similar internal representations.<n>We present a large-scale audit of convergent learning spanning dozens of vision models and thousands of layer-pair comparisons.<n>Findings fill critical gaps in our understanding of representational convergence, with implications for neuroscience and AI.
• Score: 1.9458156037869137
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Understanding convergent learning -- the degree to which independently trained neural systems -- whether multiple artificial networks or brains and models -- arrive at similar internal representations -- is crucial for both neuroscience and AI. Yet, the literature remains narrow in scope -- typically examining just a handful of models with one dataset, relying on one alignment metric, and evaluating networks at a single post-training checkpoint. We present a large-scale audit of convergent learning, spanning dozens of vision models and thousands of layer-pair comparisons, to close these long-standing gaps. First, we pit three alignment families against one another -- linear regression (affine-invariant), orthogonal Procrustes (rotation-/reflection-invariant), and permutation/soft-matching (unit-order-invariant). We find that orthogonal transformations align representations nearly as effectively as more flexible linear ones, and although permutation scores are lower, they significantly exceed chance, indicating a privileged representational basis. Tracking convergence throughout training further shows that nearly all eventual alignment crystallizes within the first epoch -- well before accuracy plateaus -- indicating it is largely driven by shared input statistics and architectural biases, not by the final task solution. Finally, when models are challenged with a battery of out-of-distribution images, early layers remain tightly aligned, whereas deeper layers diverge in proportion to the distribution shift. These findings fill critical gaps in our understanding of representational convergence, with implications for neuroscience and AI.

Related papers

• The Butterfly Effect: Neural Network Training Trajectories Are Highly Sensitive to Initial Conditions [51.68215326304272]
  We show that even small perturbations reliably cause otherwise identical training trajectories to diverge-an effect that diminishes rapidly over training time.<n>Our findings provide insights into neural network training stability, with practical implications for fine-tuning, model merging, and diversity of model ensembles.
  arXiv  Detail & Related papers  (2025-06-16T08:35:16Z)
• In-Context Linear Regression Demystified: Training Dynamics and Mechanistic Interpretability of Multi-Head Softmax Attention [52.159541540613915]
  We study how multi-head softmax attention models are trained to perform in-context learning on linear data.<n>Our results reveal that in-context learning ability emerges from the trained transformer as an aggregated effect of its architecture and the underlying data distribution.
  arXiv  Detail & Related papers  (2025-03-17T02:00:49Z)
• Pre-trained Models Succeed in Medical Imaging with Representation Similarity Degradation [6.545152478351316]
  The study establishes a rigorous problem definition centered on quantifying and analyzing representation similarity trajectories. Our empirical findings reveal the potential existence of high-performance models that preserve both task accuracy and representation similarity to their pre-trained origins.
  arXiv  Detail & Related papers  (2025-03-11T01:37:54Z)
• Non-asymptotic Convergence of Training Transformers for Next-token Prediction [48.9399496805422]
  Transformers have achieved extraordinary success in modern machine learning due to their excellent ability to handle sequential data. This paper provides a fine-grained non-asymptotic analysis of the training dynamics of a one-layer transformer. We show that the trained transformer presents non-token prediction ability with dataset shift.
  arXiv  Detail & Related papers  (2024-09-25T20:22:06Z)
• Tracing Representation Progression: Analyzing and Enhancing Layer-Wise Similarity [20.17288970927518]
  We study the similarity of representations between the hidden layers of individual transformers.<n>We show that representations across layers are positively correlated, with similarity increasing when layers get closer.<n>We propose an aligned training method to improve the effectiveness of shallow layer.
  arXiv  Detail & Related papers  (2024-06-20T16:41:09Z)
• Minimum-Norm Interpolation Under Covariate Shift [14.863831433459902]
  In-distribution research on high-dimensional linear regression has led to the identification of a phenomenon known as textitbenign overfitting We prove the first non-asymptotic excess risk bounds for benignly-overfit linear interpolators in the transfer learning setting.
  arXiv  Detail & Related papers  (2024-03-31T01:41:57Z)
• Comparing supervised learning dynamics: Deep neural networks match human data efficiency but show a generalisation lag [3.0333265803394993]
  Recent research has seen many behavioral comparisons between humans and deep neural networks (DNNs) in the domain of image classification. Here we report a detailed investigation of the learning dynamics in human observers and various classic and state-of-the-art DNNs. Across the whole learning process we evaluate and compare how well learned representations can be generalized to previously unseen test data.
  arXiv  Detail & Related papers  (2024-02-14T16:47:20Z)
• Outliers with Opposing Signals Have an Outsized Effect on Neural Network Optimization [36.72245290832128]
  We identify a new phenomenon in neural network optimization which arises from the interaction of depth and a heavytailed structure in natural data. In particular, it implies a conceptually new cause for progressive sharpening and the edge of stability. We demonstrate the significant influence of paired groups of outliers in the training data with strong opposing signals.
  arXiv  Detail & Related papers  (2023-11-07T17:43:50Z)
• In-Context Convergence of Transformers [63.04956160537308]
  We study the learning dynamics of a one-layer transformer with softmax attention trained via gradient descent. For data with imbalanced features, we show that the learning dynamics take a stage-wise convergence process.
  arXiv  Detail & Related papers  (2023-10-08T17:55:33Z)
• From Bricks to Bridges: Product of Invariances to Enhance Latent Space Communication [19.336940758147442]
  It has been observed that representations learned by distinct neural networks conceal structural similarities when the models are trained under similar inductive biases. We introduce a versatile method to directly incorporate a set of invariances into the representations, constructing a product space of invariant components on top of the latent representations. We validate our solution on classification and reconstruction tasks, observing consistent latent similarity and downstream performance improvements in a zero-shot stitching setting.
  arXiv  Detail & Related papers  (2023-10-02T13:55:38Z)
• Neural networks trained with SGD learn distributions of increasing complexity [78.30235086565388]
  We show that neural networks trained using gradient descent initially classify their inputs using lower-order input statistics. We then exploit higher-order statistics only later during training. We discuss the relation of DSB to other simplicity biases and consider its implications for the principle of universality in learning.
  arXiv  Detail & Related papers  (2022-11-21T15:27:22Z)
• Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
  We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs. By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
  arXiv  Detail & Related papers  (2022-02-01T17:11:13Z)
• Understanding Feature Transfer Through Representation Alignment [45.35473578109525]
  We find that training neural networks with different architectures and generalizations on random or true labels enforces the same relationship between the hidden representations and the training labels. We show in a classic synthetic transfer problem that alignment is the determining factor for positive and negative transfer to similar and dissimilar tasks.
  arXiv  Detail & Related papers  (2021-12-15T00:20:29Z)
• Analyzing Overfitting under Class Imbalance in Neural Networks for Image Segmentation [19.259574003403998]
  In image segmentation neural networks may overfit to the foreground samples from small structures. In this study, we provide new insights on the problem of overfitting under class imbalance by inspecting the network behavior.
  arXiv  Detail & Related papers  (2021-02-20T14:57:58Z)
• Can Temporal-Difference and Q-Learning Learn Representation? A Mean-Field Theory [110.99247009159726]
  Temporal-difference and Q-learning play a key role in deep reinforcement learning, where they are empowered by expressive nonlinear function approximators such as neural networks. In particular, temporal-difference learning converges when the function approximator is linear in a feature representation, which is fixed throughout learning, and possibly diverges otherwise.
  arXiv  Detail & Related papers  (2020-06-08T17:25:22Z)
• Revisiting Initialization of Neural Networks [72.24615341588846]
  We propose a rigorous estimation of the global curvature of weights across layers by approximating and controlling the norm of their Hessian matrix. Our experiments on Word2Vec and the MNIST/CIFAR image classification tasks confirm that tracking the Hessian norm is a useful diagnostic tool.
  arXiv  Detail & Related papers  (2020-04-20T18:12:56Z)
• Learning What Makes a Difference from Counterfactual Examples and Gradient Supervision [57.14468881854616]
  We propose an auxiliary training objective that improves the generalization capabilities of neural networks. We use pairs of minimally-different examples with different labels, a.k.a counterfactual or contrasting examples, which provide a signal indicative of the underlying causal structure of the task. Models trained with this technique demonstrate improved performance on out-of-distribution test sets.
  arXiv  Detail & Related papers  (2020-04-20T02:47:49Z)
• Kernel and Rich Regimes in Overparametrized Models [69.40899443842443]
  We show that gradient descent on overparametrized multilayer networks can induce rich implicit biases that are not RKHS norms. We also demonstrate this transition empirically for more complex matrix factorization models and multilayer non-linear networks.
  arXiv  Detail & Related papers  (2020-02-20T15:43:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.