Stable neural networks and connections to continuous dynamical systems

• URL: http://arxiv.org/abs/2510.22299v1
• Date: Sat, 25 Oct 2025 14:00:03 GMT
• Title: Stable neural networks and connections to continuous dynamical systems
• Authors: Matthias J. Ehrhardt, Davide Murari, Ferdia Sherry,
• Abstract summary: This work focuses on understanding and enhancing the stability of neural networks.<n>We identify and describe the fundamental concepts that underlie much of the existing work in this area.<n>We provide code that implements the approach that can be adapted and extended by the reader.
• Score: 0.7136933021609076
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The existence of instabilities, for example in the form of adversarial examples, has given rise to a highly active area of research concerning itself with understanding and enhancing the stability of neural networks. We focus on a popular branch within this area which draws on connections to continuous dynamical systems and optimal control, giving a bird's eye view of this area. We identify and describe the fundamental concepts that underlie much of the existing work in this area. Following this, we go into more detail on a specific approach to designing stable neural networks, developing the theoretical background and giving a description of how these networks can be implemented. We provide code that implements the approach that can be adapted and extended by the reader. The code further includes a notebook with a fleshed-out toy example on adversarial robustness of image classification that can be run without heavy requirements on the reader's computer. We finish by discussing this toy example so that the reader can interactively follow along on their computer. This work will be included as a chapter of a book on scientific machine learning, which is currently under revision and aimed at students.

Related papers

• Concept-Guided Interpretability via Neural Chunking [64.6429903327095]
  We show that neural networks exhibit patterns in their raw population activity that mirror regularities in the training data.<n>We propose three methods to extract recurring chunks on a neural population level.<n>Our work points to a new direction for interpretability, one that harnesses both cognitive principles and the structure of naturalistic data.
  arXiv  Detail & Related papers  (2025-05-16T13:49:43Z)
• Identifying Sub-networks in Neural Networks via Functionally Similar Representations [41.028797971427124]
  We take a step toward automating the understanding of the network by investigating the existence of distinct sub-networks.<n>Specifically, we explore a novel automated and task-agnostic approach based on the notion of functionally similar representations within neural networks.<n>We show the proposed approach offers meaningful insights into the behavior of neural networks with minimal human and computational cost.
  arXiv  Detail & Related papers  (2024-10-21T20:19:00Z)
• Coding schemes in neural networks learning classification tasks [52.22978725954347]
  We investigate fully-connected, wide neural networks learning classification tasks. We show that the networks acquire strong, data-dependent features. Surprisingly, the nature of the internal representations depends crucially on the neuronal nonlinearity.
  arXiv  Detail & Related papers  (2024-06-24T14:50:05Z)
• Conditional computation in neural networks: principles and research trends [48.14569369912931]
  This article summarizes principles and ideas from the emerging area of applying textitconditional computation methods to the design of neural networks. In particular, we focus on neural networks that can dynamically activate or de-activate parts of their computational graph conditionally on their input.
  arXiv  Detail & Related papers  (2024-03-12T11:56:38Z)
• DISCOVER: Making Vision Networks Interpretable via Competition and Dissection [11.028520416752325]
  This work contributes to post-hoc interpretability, and specifically Network Dissection. Our goal is to present a framework that makes it easier to discover the individual functionality of each neuron in a network trained on a vision task.
  arXiv  Detail & Related papers  (2023-10-07T21:57:23Z)
• Stochastic Coherence Over Attention Trajectory For Continuous Learning In Video Streams [64.82800502603138]
  This paper proposes a novel neural-network-based approach to progressively and autonomously develop pixel-wise representations in a video stream. The proposed method is based on a human-like attention mechanism that allows the agent to learn by observing what is moving in the attended locations. Our experiments leverage 3D virtual environments and they show that the proposed agents can learn to distinguish objects just by observing the video stream.
  arXiv  Detail & Related papers  (2022-04-26T09:52:31Z)
• Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
  We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs. By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
  arXiv  Detail & Related papers  (2022-02-01T17:11:13Z)
• Learning Contact Dynamics using Physically Structured Neural Networks [81.73947303886753]
  We use connections between deep neural networks and differential equations to design a family of deep network architectures for representing contact dynamics between objects. We show that these networks can learn discontinuous contact events in a data-efficient manner from noisy observations. Our results indicate that an idealised form of touch feedback is a key component of making this learning problem tractable.
  arXiv  Detail & Related papers  (2021-02-22T17:33:51Z)
• Analyzing Representations inside Convolutional Neural Networks [8.803054559188048]
  We propose a framework to categorize the concepts a network learns based on the way it clusters a set of input examples. This framework is unsupervised and can work without any labels for input features. We extensively evaluate the proposed method and demonstrate that it produces human-understandable and coherent concepts.
  arXiv  Detail & Related papers  (2020-12-23T07:10:17Z)
• How Researchers Use Diagrams in Communicating Neural Network Systems [5.064404027153093]
  This paper reports on a study into the use of neural network system diagrams. We find high diversity of usage, perception and preference in both creation and interpretation of diagrams. Considering the interview data alongside existing guidance, we propose guidelines aiming to improve the way in which neural network system diagrams are constructed.
  arXiv  Detail & Related papers  (2020-08-28T10:21:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.