Clustering and Alignment: Understanding the Training Dynamics in Modular Addition

• URL: http://arxiv.org/abs/2408.09414v2
• Date: Sun, 27 Oct 2024 21:40:33 GMT
• Title: Clustering and Alignment: Understanding the Training Dynamics in Modular Addition
• Authors: Tiberiu Musat,
• Abstract summary: I study the training dynamics of a small neural network with 2-dimensional embeddings on the problem of modular addition. I study these structures and explain their emergence as a result of two simple tendencies exhibited by pairs of embeddings. I discuss the role of weight decay in my setup and reveal a new mechanism that links regularization and training dynamics.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent studies have revealed that neural networks learn interpretable algorithms for many simple problems. However, little is known about how these algorithms emerge during training. In this article, I study the training dynamics of a small neural network with 2-dimensional embeddings on the problem of modular addition. I observe that embedding vectors tend to organize into two types of structures: grids and circles. I study these structures and explain their emergence as a result of two simple tendencies exhibited by pairs of embeddings: clustering and alignment. I propose explicit formulae for these tendencies as interaction forces between different pairs of embeddings. To show that my formulae can fully account for the emergence of these structures, I construct an equivalent particle simulation where I show that identical structures emerge. I discuss the role of weight decay in my setup and reveal a new mechanism that links regularization and training dynamics. To support my findings, I also release an interactive demo available at https://modular-addition.vercel.app/.

Related papers

• From Dionysius Emerges Apollo -- Learning Patterns and Abstractions from Perceptual Sequences [1.3597551064547502]
  A sensory stream, simplified, is a one-dimensional sequence. In learning such sequences, we naturally segment them into parts -- a process known as chunking. I developed models that learn chunks and parse sequences chunk by chunk.
  arXiv  Detail & Related papers  (2025-03-14T00:37:28Z)
• Modular Training of Neural Networks aids Interpretability [45.8172254436063]
  We define a measure for clusterability and show that pre-trained models form highly enmeshed clusters via spectral graph clustering. Using automated interpretability techniques, we show that our method can help train models that are more modular and learn different, disjoint, and smaller circuits.
  arXiv  Detail & Related papers  (2025-02-04T16:44:38Z)
• Structure Development in List-Sorting Transformers [0.0]
  We study how a one-layer attention-only transformer develops relevant structures while learning to sort lists of numbers. At the end of training, the model organizes its attention heads in two main modes that we refer to as vocabulary-splitting and copy-suppression.
  arXiv  Detail & Related papers  (2025-01-30T15:56:25Z)
• Neural Metamorphosis [72.88137795439407]
  This paper introduces a new learning paradigm termed Neural Metamorphosis (NeuMeta), which aims to build self-morphable neural networks. NeuMeta directly learns the continuous weight manifold of neural networks. It sustains full-size performance even at a 75% compression rate.
  arXiv  Detail & Related papers  (2024-10-10T14:49:58Z)
• Seeing is Believing: Brain-Inspired Modular Training for Mechanistic Interpretability [5.15188009671301]
  Brain-Inspired Modular Training is a method for making neural networks more modular and interpretable. BIMT embeds neurons in a geometric space and augments the loss function with a cost proportional to the length of each neuron connection.
  arXiv  Detail & Related papers  (2023-05-04T17:56:42Z)
• How Do Transformers Learn Topic Structure: Towards a Mechanistic Understanding [56.222097640468306]
  We provide mechanistic understanding of how transformers learn "semantic structure" We show, through a combination of mathematical analysis and experiments on Wikipedia data, that the embedding layer and the self-attention layer encode the topical structure.
  arXiv  Detail & Related papers  (2023-03-07T21:42:17Z)
• Unsupervised Learning of Equivariant Structure from Sequences [30.974508897223124]
  We present an unsupervised framework to learn the symmetry from the time sequence of length at least three. We will demonstrate that, with our framework, the hidden disentangled structure of the dataset naturally emerges as a by-product.
  arXiv  Detail & Related papers  (2022-10-12T07:29:18Z)
• Clustering units in neural networks: upstream vs downstream information [3.222802562733787]
  We study modularity of hidden layer representations of feedforward, fully connected networks. We find two surprising results: first, dropout dramatically increased modularity, while other forms of weight regularization had more modest effects. This has important implications for representation-learning, as it suggests that finding modular representations that reflect structure in inputs may be a distinct goal from learning modular representations that reflect structure in outputs.
  arXiv  Detail & Related papers  (2022-03-22T15:35:10Z)
• Graph Kernel Neural Networks [53.91024360329517]
  We propose to use graph kernels, i.e. kernel functions that compute an inner product on graphs, to extend the standard convolution operator to the graph domain. This allows us to define an entirely structural model that does not require computing the embedding of the input graph. Our architecture allows to plug-in any type of graph kernels and has the added benefit of providing some interpretability.
  arXiv  Detail & Related papers  (2021-12-14T14:48:08Z)
• Dynamic Inference with Neural Interpreters [72.90231306252007]
  We present Neural Interpreters, an architecture that factorizes inference in a self-attention network as a system of modules. inputs to the model are routed through a sequence of functions in a way that is end-to-end learned. We show that Neural Interpreters perform on par with the vision transformer using fewer parameters, while being transferrable to a new task in a sample efficient manner.
  arXiv  Detail & Related papers  (2021-10-12T23:22:45Z)
• S2RMs: Spatially Structured Recurrent Modules [105.0377129434636]
  We take a step towards exploiting dynamic structure that are capable of simultaneously exploiting both modular andtemporal structures. We find our models to be robust to the number of available views and better capable of generalization to novel tasks without additional training.
  arXiv  Detail & Related papers  (2020-07-13T17:44:30Z)
• Learning compositional functions via multiplicative weight updates [97.9457834009578]
  We show that multiplicative weight updates satisfy a descent lemma tailored to compositional functions. We show that Madam can train state of the art neural network architectures without learning rate tuning.
  arXiv  Detail & Related papers  (2020-06-25T17:05:19Z)
• Pruned Neural Networks are Surprisingly Modular [9.184659875364689]
  We introduce a measurable notion of modularity for multi-layer perceptrons. We investigate the modular structure of neural networks trained on datasets of small images.
  arXiv  Detail & Related papers  (2020-03-10T17:51:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.