Modular connectivity in neural networks emerges from Poisson noise-motivated regularisation, and promotes robustness and compositional generalisation

• URL: http://arxiv.org/abs/2512.13707v1
• Date: Fri, 05 Dec 2025 20:20:36 GMT
• Title: Modular connectivity in neural networks emerges from Poisson noise-motivated regularisation, and promotes robustness and compositional generalisation
• Authors: Daoyuan Qian, Qiyao Liang, Ila Fiete,
• Abstract summary: Circuits in the brain commonly exhibit modular architectures that factorise complex tasks.<n>In contrast, artificial neural networks (ANNs) appear to mix all processing.<n>We show that activity-dependent neural noise, combined with nonlinear neural responses, drives the emergence of solutions.
• Score: 9.273764753674177
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Circuits in the brain commonly exhibit modular architectures that factorise complex tasks, resulting in the ability to compositionally generalise and reduce catastrophic forgetting. In contrast, artificial neural networks (ANNs) appear to mix all processing, because modular solutions are difficult to find as they are vanishing subspaces in the space of possible solutions. Here, we draw inspiration from fault-tolerant computation and the Poisson-like firing of real neurons to show that activity-dependent neural noise, combined with nonlinear neural responses, drives the emergence of solutions that reflect an accurate understanding of modular tasks, corresponding to acquisition of a correct world model. We find that noise-driven modularisation can be recapitulated by a deterministic regulariser that multiplicatively combines weights and activations, revealing rich phenomenology not captured in linear networks or by standard regularisation methods. Though the emergence of modular structure requires sufficiently many training samples (exponential in the number of modular task dimensions), we show that pre-modularised ANNs exhibit superior noise-robustness and the ability to generalise and extrapolate well beyond training data, compared to ANNs without such inductive biases. Together, our work demonstrates a regulariser and architectures that could encourage modularity emergence to yield functional benefits.

Related papers

• A Theoretical Framework for Modular Learning of Robust Generative Models [41.69461814486466]
  Training large-scale generative models is resource-intensive and relies heavily on dataset weighting.<n>We present a theoretical framework for modular generative modeling where a set of pre-trained experts are combined via a gating mechanism.<n>We show that our modular architecture effectively mitigates conflict and can robustly outperform monolithic baselines.
  arXiv  Detail & Related papers  (2026-02-19T17:09:13Z)
• Quantum LEGO Learning: A Modular Design Principle for Hybrid Artificial Intelligence [63.39968536637762]
  We introduce Quantum LEGO Learning, a learning framework that treats classical and quantum components as reusable, composable learning blocks.<n>Within this framework, a pre-trained classical neural network serves as a frozen feature block, while a VQC acts as a trainable adaptive module.<n>We develop a block-wise generalization theory that decomposes learning error into approximation and estimation components.
  arXiv  Detail & Related papers  (2026-01-29T14:29:21Z)
• Inducing, Detecting and Characterising Neural Modules: A Pipeline for Functional Interpretability in Reinforcement Learning [1.597617022056624]
  We show how encouraging sparsity and locality in network weights leads to the emergence of functional modules in RL policy networks.<n>Applying these methods to 2D and 3D MiniGrid environments reveals the consistent emergence of distinct navigational modules for different axes.
  arXiv  Detail & Related papers  (2025-01-28T17:02:16Z)
• Range, not Independence, Drives Modularity in Biologically Inspired Representations [52.48094670415497]
  We develop a theory of when biologically inspired networks modularise their representation of source variables (sources)<n>We derive necessary and sufficient conditions on a sample of sources that determine whether the neurons in an optimal linear autoencoder modularise.<n>Our theory applies to any dataset, extending far beyond the case of statistical independence studied in previous work.
  arXiv  Detail & Related papers  (2024-10-08T17:41:37Z)
• Breaking Neural Network Scaling Laws with Modularity [8.482423139660153]
  We show how the amount of training data required to generalize varies with the intrinsic dimensionality of a task's input.<n>We then develop a novel learning rule for modular networks to exploit this advantage.
  arXiv  Detail & Related papers  (2024-09-09T16:43:09Z)
• Modular Growth of Hierarchical Networks: Efficient, General, and Robust Curriculum Learning [0.0]
  We show that for a given classical, non-modular recurrent neural network (RNN), an equivalent modular network will perform better across multiple metrics. We demonstrate that the inductive bias introduced by the modular topology is strong enough for the network to perform well even when the connectivity within modules is fixed. Our findings suggest that gradual modular growth of RNNs could provide advantages for learning increasingly complex tasks on evolutionary timescales.
  arXiv  Detail & Related papers  (2024-06-10T13:44:07Z)
• Phase codes emerge in recurrent neural networks optimized for modular arithmetic [0.0]
  We investigate whether phase codes can emerge purely from task optimization by training continuous-time RNNs to perform a simple arithmetic task.<n>We find that in the absence of such biases, RNNs can learn phase code solutions.<n>These results suggest that phase coding can be a natural but not inevitable outcome of training RNNs on modular arithmetic.
  arXiv  Detail & Related papers  (2023-10-11T21:25:12Z)
• A Deep Unrolling Model with Hybrid Optimization Structure for Hyperspectral Image Deconvolution [50.13564338607482]
  We propose a novel optimization framework for the hyperspectral deconvolution problem, called DeepMix.<n>It consists of three distinct modules, namely, a data consistency module, a module that enforces the effect of the handcrafted regularizers, and a denoising module.<n>This work proposes a context aware denoising module designed to sustain the advancements achieved by the cooperative efforts of the other modules.
  arXiv  Detail & Related papers  (2023-06-10T08:25:16Z)
• Modular Deep Learning [120.36599591042908]
  Transfer learning has recently become the dominant paradigm of machine learning. It remains unclear how to develop models that specialise towards multiple tasks without incurring negative interference. Modular deep learning has emerged as a promising solution to these challenges.
  arXiv  Detail & Related papers  (2023-02-22T18:11:25Z)
• On the Generalization and Adaption Performance of Causal Models [99.64022680811281]
  Differentiable causal discovery has proposed to factorize the data generating process into a set of modules. We study the generalization and adaption performance of such modular neural causal models. Our analysis shows that the modular neural causal models outperform other models on both zero and few-shot adaptation in low data regimes.
  arXiv  Detail & Related papers  (2022-06-09T17:12:32Z)
• Towards Understanding the Link Between Modularity and Performance in Neural Networks for Reinforcement Learning [2.038038953957366]
  We find that the amount of network modularity for optimal performance is likely entangled in complex relationships between many other features of the network and problem environment. We used a classic neuroevolutionary algorithm which enables rich, automatic optimisation and exploration of neural network architectures.
  arXiv  Detail & Related papers  (2022-05-13T05:18:18Z)
• S2RMs: Spatially Structured Recurrent Modules [105.0377129434636]
  We take a step towards exploiting dynamic structure that are capable of simultaneously exploiting both modular andtemporal structures. We find our models to be robust to the number of available views and better capable of generalization to novel tasks without additional training.
  arXiv  Detail & Related papers  (2020-07-13T17:44:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.