Architecture Generalization with MetaNCA

• URL: http://arxiv.org/abs/2607.07743v1
• Date: Wed, 08 Jul 2026 08:31:57 GMT
• Title: Architecture Generalization with MetaNCA
• Abstract summary: We introduce Meta Neural Cellular Automata (MetaNCA), a framework that learns local rules which self-organize the weights of artificial neural networks.<n>We show that MetaNCA generates weights for feedforwards, CNNs, and ResNets on MNIST and CIFAR-100, scaling to networks of 2 million parameters.
• Score: 1.2371991034712437
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Self-organization is an emergent property of life, driven by the collective behavior of individual components acting on local information. Biological neurons, through local interactions transmitted through synapses, are able to learn efficiently and can adapt their connections over an organism's lifespan. Motivated by these desirable properties of adaptability and local interaction, neural cellular automata (NCA) models have been successful at learning morphogenesis solely through local update rules, demonstrating stability over many updates and robustness to perturbations. In this work, we introduce Meta Neural Cellular Automata (MetaNCA), a framework that learns local rules which self-organize the weights of artificial neural networks. A learned rule network iteratively updates the weights of a task network using only local interactions on the computation graph. We propose a novel Weight Transformer architecture for the local rule network, which uses linear attention to aggregate signals from neighboring weights and hidden states. Once trained, the rule network generates task networks of diverse architectures without backpropagation. We show that MetaNCA generates weights for feedforward MLPs, CNNs, and ResNets on MNIST and CIFAR-100, scaling to networks of 2 million parameters. We further show that MetaNCA generalizes to architectures not seen during meta-training, and that architectural diversity in the training phase strengthens this generalization.

Related papers

• Scalable Learning in Structured Recurrent Spiking Neural Networks without Backpropagation [6.572000338549628]
  Spiking Neural Networks (SNNs) provide a promising framework for energy-efficient and biologically grounded computation.<n>We propose a structured multi-layer recurrent SNN architecture composed of locally dense recurrent layers augmented with sparse small-world long-range projections to a readout population.<n>The long-range connectivity is largely fixed, preserving routing efficiency and hardware scalability, while synaptic adaptation is performed using strictly local plasticity mechanisms.
  arXiv  Detail & Related papers  (2026-05-01T04:45:21Z)
• NNiT: Width-Agnostic Neural Network Generation with Structurally Aligned Weight Spaces [15.631276865948097]
  We introduce Neural Network Diffusion Transformers (NNiTs), which generate weights in a width-agnostic manner.<n>On ManiSkill3 robotics tasks, NNiT achieves >85% success on architecture topologies unseen during training, while baseline approaches fail to generalize.
  arXiv  Detail & Related papers  (2026-02-26T20:47:30Z)
• Self-Motivated Growing Neural Network for Adaptive Architecture via Local Structural Plasticity [0.2578242050187029]
  Control policies in deep reinforcement learning are often implemented with fixed-capacity multilayer perceptrons trained by backpropagation.<n>This paper introduces the Self-Motivated Growing Neural Network (SMGrNN), a controller whose topology evolves online through a local Structural Plasticity Module.
  arXiv  Detail & Related papers  (2025-12-14T14:31:21Z)
• Structural Plasticity as Active Inference: A Biologically-Inspired Architecture for Homeostatic Control [0.0]
  This paper introduces the Structurally Adaptive Predictive Inference Network (SAPIN)<n> SAPIN operates on a 2D grid where cells learn by minimizing local prediction errors.<n>We validated the SAPIN model on the classic Cart Pole reinforcement learning benchmark.
  arXiv  Detail & Related papers  (2025-11-04T04:07:16Z)
• Auto-Compressing Networks [51.221103189527014]
  We introduce Auto-compression Networks (ACNs), an architectural variant where long feedforward connections from each layer replace traditional short residual connections.<n>We show that ACNs exhibit enhanced noise compared to residual networks, superior performance in low-data settings, and mitigate catastrophic forgetting.<n>These findings establish ACNs as a practical approach to developing efficient neural architectures.
  arXiv  Detail & Related papers  (2025-06-11T13:26:09Z)
• Generalization emerges from local optimization in a self-organized learning network [0.0]
  We design and analyze a new paradigm for building supervised learning networks, driven only by local optimization rules without relying on a global error function. Our network stores new knowledge in the nodes accurately and instantaneously, in the form of a lookup table. We show on numerous examples of classification tasks that the networks generated by our algorithm systematically reach such a state of perfect generalization when the number of learned examples becomes sufficiently large. We report on the dynamics of the change of state and show that it is abrupt and has the distinctive characteristics of a first order phase transition, a phenomenon already observed for traditional learning networks and known as grokking.
  arXiv  Detail & Related papers  (2024-10-03T15:32:08Z)
• How neural networks learn to classify chaotic time series [77.34726150561087]
  We study the inner workings of neural networks trained to classify regular-versus-chaotic time series. We find that the relation between input periodicity and activation periodicity is key for the performance of LKCNN models.
  arXiv  Detail & Related papers  (2023-06-04T08:53:27Z)
• PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning [109.84770951839289]
  We present PredRNN, a new recurrent network for learning visual dynamics from historical context. We show that our approach obtains highly competitive results on three standard datasets.
  arXiv  Detail & Related papers  (2021-03-17T08:28:30Z)
• Firefly Neural Architecture Descent: a General Approach for Growing Neural Networks [50.684661759340145]
  Firefly neural architecture descent is a general framework for progressively and dynamically growing neural networks. We show that firefly descent can flexibly grow networks both wider and deeper, and can be applied to learn accurate but resource-efficient neural architectures. In particular, it learns networks that are smaller in size but have higher average accuracy than those learned by the state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-02-17T04:47:18Z)
• Neural Cellular Automata Manifold [84.08170531451006]
  We show that the neural network architecture of the Neural Cellular Automata can be encapsulated in a larger NN. This allows us to propose a new model that encodes a manifold of NCA, each of them capable of generating a distinct image. In biological terms, our approach would play the role of the transcription factors, modulating the mapping of genes into specific proteins that drive cellular differentiation.
  arXiv  Detail & Related papers  (2020-06-22T11:41:57Z)
• Neuroevolutionary Transfer Learning of Deep Recurrent Neural Networks through Network-Aware Adaptation [57.46377517266827]
  This work introduces network-aware adaptive structure transfer learning (N-ASTL) N-ASTL utilizes statistical information related to the source network's topology and weight distribution to inform how new input and output neurons are to be integrated into the existing structure. Results show improvements over prior state-of-the-art, including the ability to transfer in challenging real-world datasets not previously possible.
  arXiv  Detail & Related papers  (2020-06-04T06:07:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.