Training Deep Visual Networks Beyond Loss and Accuracy Through a Dynamical Systems Approach

• URL: http://arxiv.org/abs/2604.09716v1
• Date: Wed, 08 Apr 2026 12:41:10 GMT
• Title: Training Deep Visual Networks Beyond Loss and Accuracy Through a Dynamical Systems Approach
• Abstract summary: This paper introduces a complementary way to study that process by examining training through the lens of dynamical systems.<n>We define three measures from layer activations collected across training epochs.<n>We apply this framework to nine combinations of model architecture and dataset.
• Score: 0.6229352230128421
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep visual recognition models are usually trained and evaluated using metrics such as loss and accuracy. While these measures show whether a model is improving, they reveal very little about how its internal representations change during training. This paper introduces a complementary way to study that process by examining training through the lens of dynamical systems. Drawing on ideas from signal analysis originally used to study biological neural activity, we define three measures from layer activations collected across training epochs: an integration score that reflects long-range coordination across layers, a metastability score that captures how flexibly the network shifts between more and less synchronised states, and a combined dynamical stability index. We apply this framework to nine combinations of model architecture and dataset, including several ResNet variants, DenseNet-121, MobileNetV2, VGG-16, and a pretrained Vision Transformer on CIFAR-10 and CIFAR-100. The results suggest three main patterns. First, the integration measure consistently distinguishes the easier CIFAR-10 setting from the more difficult CIFAR-100 setting. Second, changes in the volatility of the stability index may provide an early sign of convergence before accuracy fully plateaus. Third, the relationship between integration and metastability appears to reflect different styles of training behaviour. Overall, this study offers an exploratory but promising new way to understand deep visual training beyond loss and accuracy.

Related papers

• Boosting Robust AIGI Detection with LoRA-based Pairwise Training [55.076681464804636]
  Current AIGI detectors perform admirably on clean datasets, but their detection performance decreases when deployed "in the wild"<n>We propose a novel Lo-based Pairwise Training (RA) strategy designed specifically to achieve robust detection for AIGI under severe distortions.
  arXiv  Detail & Related papers  (2026-04-14T05:35:32Z)
• GradAttn: Replacing Fixed Residual Connections with Task-Modulated Attention Pathways [2.105564340986074]
  GradAttn is a CNN-transformer framework that replaces fixed residual connections with attention-controlled gradient flow.<n>For representational analysis, we evaluated three GradAttn variants across eight diverse datasets.<n>Results demonstrate that GradAttn outperforms ResNet-18 on five of eight datasets.
  arXiv  Detail & Related papers  (2026-03-23T14:45:07Z)
• Implicit Neural Representation-Based Continuous Single Image Super Resolution: An Empirical Study [50.15623093332659]
  Implicit neural representation (INR) has become the standard approach for arbitrary-scale image super-resolution (ASSR)<n>We compare existing techniques across diverse settings and present aggregated performance results on multiple image quality metrics.<n>We examine a new loss function that penalizes intensity variations while preserving edges, textures, and finer details during training.
  arXiv  Detail & Related papers  (2026-01-25T07:09:20Z)
• Rethinking the Role of Dynamic Sparse Training for Scalable Deep Reinforcement Learning [58.533203990515034]
  Scaling neural networks has driven breakthrough advances in machine learning, yet this paradigm fails in deep reinforcement learning (DRL)<n>We show that dynamic sparse training strategies provide module-specific benefits that complement the primary scalability foundation established by architectural improvements.<n>We finally distill these insights into Module-Specific Training (MST), a practical framework that exploits the benefits of architectural improvements and demonstrates substantial scalability gains across diverse RL algorithms without algorithmic modifications.
  arXiv  Detail & Related papers  (2025-10-14T03:03:08Z)
• Dynamic Adaptive Parsing of Temporal and Cross-Variable Patterns for Network State Classification [9.007194066331452]
  We introduce DAPNet, a framework based on a Mixture-of-Experts architecture.<n>DAPNet integrates three specialized networks for periodic analysis, dynamic cross-variable correlation modeling, and hybrid temporal feature extraction.<n>The generalizability of the architectural design is evaluated across ten public UEA benchmark datasets.
  arXiv  Detail & Related papers  (2025-09-15T05:32:32Z)
• Unleashing Network Potentials for Semantic Scene Completion [50.95486458217653]
  This paper proposes a novel SSC framework - Adrial Modality Modulation Network (AMMNet) AMMNet introduces two core modules: a cross-modal modulation enabling the interdependence of gradient flows between modalities, and a customized adversarial training scheme leveraging dynamic gradient competition. Extensive experimental results demonstrate that AMMNet outperforms state-of-the-art SSC methods by a large margin.
  arXiv  Detail & Related papers  (2024-03-12T11:48:49Z)
• Harnessing Orthogonality to Train Low-Rank Neural Networks [0.07538606213726905]
  This study explores the learning dynamics of neural networks by analyzing the singular value decomposition (SVD) of their weights throughout training. We introduce Orthogonality-Informed Adaptive Low-Rank (OIALR) training, a novel training method exploiting the intrinsic orthogonality of neural networks.
  arXiv  Detail & Related papers  (2024-01-16T17:07:22Z)
• Adaptive Multi-step Refinement Network for Robust Point Cloud Registration [82.64560249066734]
  Point Cloud Registration estimates the relative rigid transformation between two point clouds of the same scene.<n>We propose an adaptive multi-step refinement network that refines the registration quality at each step by leveraging the information from the preceding step.<n>Our method achieves state-of-the-art performance on both the 3DMatch/3DLoMatch and KITTI benchmarks.
  arXiv  Detail & Related papers  (2023-12-05T18:59:41Z)
• Analyzing and Improving the Training Dynamics of Diffusion Models [36.37845647984578]
  We identify and rectify several causes for uneven and ineffective training in the popular ADM diffusion model architecture. We find that systematic application of this philosophy eliminates the observed drifts and imbalances, resulting in considerably better networks at equal computational complexity.
  arXiv  Detail & Related papers  (2023-12-05T11:55:47Z)
• A Generic Shared Attention Mechanism for Various Backbone Neural Networks [53.36677373145012]
  Self-attention modules (SAMs) produce strongly correlated attention maps across different layers. Dense-and-Implicit Attention (DIA) shares SAMs across layers and employs a long short-term memory module. Our simple yet effective DIA can consistently enhance various network backbones.
  arXiv  Detail & Related papers  (2022-10-27T13:24:08Z)
• PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning [109.84770951839289]
  We present PredRNN, a new recurrent network for learning visual dynamics from historical context. We show that our approach obtains highly competitive results on three standard datasets.
  arXiv  Detail & Related papers  (2021-03-17T08:28:30Z)
• Training Sparse Neural Networks using Compressed Sensing [13.84396596420605]
  We develop and test a novel method based on compressed sensing which combines the pruning and training into a single step. Specifically, we utilize an adaptively weighted $ell1$ penalty on the weights during training, which we combine with a generalization of the regularized dual averaging (RDA) algorithm in order to train sparse neural networks.
  arXiv  Detail & Related papers  (2020-08-21T19:35:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.