Structure Over Nonlinearity: Explicit Interaction Architectures for Dynamical Learning

• URL: http://arxiv.org/abs/2606.19101v1
• Date: Wed, 17 Jun 2026 14:12:27 GMT
• Title: Structure Over Nonlinearity: Explicit Interaction Architectures for Dynamical Learning
• Abstract summary: We introduce a class of explicit structured dynamical units based on wave-inspired interaction structures with internal state.<n>Inspired by wave-based computational principles, the proposed units adopt a strictly causal organization.<n>We show that depth improves representation quality and generalization, even under limited parameter optimization.
• Score: 9.644668801623192
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Most learning architectures for dynamical systems rely on generic nonlinear function approximation, often requiring high model complexity to capture structured behaviors. In this work, we propose an alternative paradigm in which modeling capability arises primarily from structure rather than from expressive nonlinearities. We introduce a class of explicit structured dynamical units based on wave-inspired interaction structures with internal state. Inspired by wave-based computational principles, the proposed units adopt a strictly causal organization that eliminates algebraic loops, yielding fully explicit models that can be evaluated without implicit solvers. Stacking such units produces layered dynamical architectures with emergent hierarchical behavior. Through experiments on a nonlinear system identification task, we show that depth improves both representation quality and generalization, even under limited parameter optimization. In particular, the proposed architectures produce informative internal representations even under readout-only fitting, indicating that useful dynamical structure emerges from the organization of interactions prior to substantial parameter optimization. These results suggest that structure-first design provides a viable and effective alternative to conventional black-box approaches for learning dynamical systems, highlighting the role of interaction structure as a primary source of model expressivity.

Related papers

• A Unified Framework for Structured Flow Modeling: From Continuous Fields to Data-Driven Representations [0.0]
  dynamical systems can be described in terms of structured flows combining source/sink behavior, cyclic dynamics, and topology-constrained transport.<n>This work provides a unified perspective on such systems by connecting continuous formulations based on the Helmholtz-Hodge decomposition with discrete and data-driven representations.
  arXiv  Detail & Related papers  (2026-05-18T11:52:09Z)
• TopoOR: A Unified Topological Scene Representation for the Operating Room [54.50897207203292]
  Surgical Scene Graphs abstract the complexity of surgical operating rooms (OR) into a structure of entities and their relations.<n>We introduce TopoOR, a new paradigm that models multimodal operating rooms as a higher-order structure.<n>We also propose a higher-order attention mechanism that explicitly preserves manifold structure and modality-specific features.
  arXiv  Detail & Related papers  (2026-03-10T10:19:42Z)
• ExpertWeaver: Unlocking the Inherent MoE in Dense LLMs with GLU Activation Patterns [68.61814799047956]
  Mixture-of-Experts (MoE) effectively scales model capacity while preserving computational efficiency through sparse expert activation.<n>We introduce ExpertWeaver, a training-free framework that partitions neurons according to their activation patterns and constructs shared experts and specialized routed experts with layer-adaptive configurations.
  arXiv  Detail & Related papers  (2026-02-17T11:50:58Z)
• Structural Disentanglement in Bilinear MLPs via Architectural Inductive Bias [0.0]
  We argue that failures arise from how models structure their internal representations during training.<n>We show analytically that bilinear parameterizations possess a non-mixing' property under gradient flow conditions.<n>Unlike pointwise nonlinear networks, multiplicative architectures are able to recover true operators aligned with the underlying algebraic structure.
  arXiv  Detail & Related papers  (2026-02-05T13:14:01Z)
• From Coefficients to Directions: Rethinking Model Merging with Directional Alignment [66.99062575537555]
  We introduce a unified geometric framework, emphMerging with Directional Alignment (method), which aligns directional structures consistently in both the parameter and feature spaces.<n>Our analysis shows that directional alignment improves structural coherence, and extensive experiments across benchmarks, model scales, and task configurations further validate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2025-11-29T08:40:58Z)
• Rethinking the Role of Dynamic Sparse Training for Scalable Deep Reinforcement Learning [58.533203990515034]
  Scaling neural networks has driven breakthrough advances in machine learning, yet this paradigm fails in deep reinforcement learning (DRL)<n>We show that dynamic sparse training strategies provide module-specific benefits that complement the primary scalability foundation established by architectural improvements.<n>We finally distill these insights into Module-Specific Training (MST), a practical framework that exploits the benefits of architectural improvements and demonstrates substantial scalability gains across diverse RL algorithms without algorithmic modifications.
  arXiv  Detail & Related papers  (2025-10-14T03:03:08Z)
• Information-Theoretic Bounds and Task-Centric Learning Complexity for Real-World Dynamic Nonlinear Systems [0.6875312133832079]
  Dynamic nonlinear systems exhibit distortions arising from coupled static and dynamic effects.<n>This paper presents a theoretical framework grounded in structured decomposition, variance analysis, and task-centric complexity bounds.
  arXiv  Detail & Related papers  (2025-09-08T12:08:02Z)
• Understanding Learning Dynamics Through Structured Representations [1.7244210453129227]
  This paper investigates how internal structural choices shape the behavior of learning systems.<n>We analyze how these structures influence gradient flow, spectral sensitivity, and fixed-point behavior.<n>Rather than prescribing fixed templates, we emphasize principles of tractable design that can steer learning behavior in interpretable ways.
  arXiv  Detail & Related papers  (2025-08-04T07:15:57Z)
• Modeling Hierarchical Spaces: A Review and Unified Framework for Surrogate-Based Architecture Design [0.32622301272834514]
  This paper reviews literature on structured input spaces and proposes a unified framework that generalizes existing approaches.<n>A variable is described as meta if its value governs the presence of other decreed variables, enabling the modeling of conditional and hierarchical structures.<n>Our framework defines hierarchical distances and kernels to enable surrogate modeling and optimization on hierarchical domains.
  arXiv  Detail & Related papers  (2025-06-27T20:38:57Z)
• Generalized Factor Neural Network Model for High-dimensional Regression [50.554377879576066]
  We tackle the challenges of modeling high-dimensional data sets with latent low-dimensional structures hidden within complex, non-linear, and noisy relationships.<n>Our approach enables a seamless integration of concepts from non-parametric regression, factor models, and neural networks for high-dimensional regression.
  arXiv  Detail & Related papers  (2025-02-16T23:13:55Z)
• Generalized Quadratic Embeddings for Nonlinear Dynamics using Deep Learning [11.339982217541822]
  We present a data-driven methodology for modeling the dynamics of nonlinear systems. In this work, we propose using quadratic systems as the common structure, inspired by the lifting principle.
  arXiv  Detail & Related papers  (2022-11-01T10:03:34Z)
• Structure-Regularized Attention for Deformable Object Representation [17.120035855774344]
  Capturing contextual dependencies has proven useful to improve the representational power of deep neural networks. Recent approaches that focus on modeling global context, such as self-attention and non-local operation, achieve this goal by enabling unconstrained pairwise interactions between elements. We consider learning representations for deformable objects which can benefit from context exploitation by modeling the structural dependencies that the data intrinsically possesses.
  arXiv  Detail & Related papers  (2021-06-12T03:10:17Z)
• S2RMs: Spatially Structured Recurrent Modules [105.0377129434636]
  We take a step towards exploiting dynamic structure that are capable of simultaneously exploiting both modular andtemporal structures. We find our models to be robust to the number of available views and better capable of generalization to novel tasks without additional training.
  arXiv  Detail & Related papers  (2020-07-13T17:44:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.