Hierarchical-embedding autoencoder with a predictor (HEAP) as efficient architecture for learning long-term evolution of complex multi-scale physical systems

• URL: http://arxiv.org/abs/2505.18857v1
• Date: Sat, 24 May 2025 20:27:16 GMT
• Title: Hierarchical-embedding autoencoder with a predictor (HEAP) as efficient architecture for learning long-term evolution of complex multi-scale physical systems
• Authors: Alexander Khrabry, Edward Startsev, Andrew Powis, Igor Kaganovich,
• Abstract summary: Structures of various scales that dynamically emerge in the system interact with each other only locally.<n>The hierarchical fully-convolutional autoencoder transforms the state of a physical system into a series of embedding layers.<n> Interactions between features of various scales are modeled using a combination of convolutional operators.
• Score: 41.94295877935867
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: We propose a novel efficient architecture for learning long-term evolution in complex multi-scale physical systems which is based on the idea of separation of scales. Structures of various scales that dynamically emerge in the system interact with each other only locally. Structures of similar scale can interact directly when they are in contact and indirectly when they are parts of larger structures that interact directly. This enables modeling a multi-scale system in an efficient way, where interactions between small-scale features that are apart from each other do not need to be modeled. The hierarchical fully-convolutional autoencoder transforms the state of a physical system not just into a single embedding layer, as it is done conventionally, but into a series of embedding layers which encode structures of various scales preserving spatial information at a corresponding resolution level. Shallower layers embed smaller structures on a finer grid, while deeper layers embed larger structures on a coarser grid. The predictor advances all embedding layers in sync. Interactions between features of various scales are modeled using a combination of convolutional operators. We compare the performance of our model to variations of a conventional ResNet architecture in application to the Hasegawa-Wakatani turbulence. A multifold improvement in long-term prediction accuracy was observed for crucial statistical characteristics of this system.

Related papers

• Broad Spectrum Structure Discovery in Large-Scale Higher-Order Networks [1.7273380623090848]
  We introduce a class of probabilistic models that efficiently represents and discovers a broad spectrum of mesoscale structure in large-scale hypergraphs.<n>By modeling observed node interactions through latent interactions among classes using low-rank representations, our approach tractably captures rich structural patterns.<n>Our model improves link prediction over state-of-the-art methods and discovers interpretable structures in diverse real-world systems.
  arXiv  Detail & Related papers  (2025-05-27T20:34:58Z)
• An Efficient and Mixed Heterogeneous Model for Image Restoration [71.85124734060665]
  Current mainstream approaches are based on three architectural paradigms: CNNs, Transformers, and Mambas.<n>We propose RestorMixer, an efficient and general-purpose IR model based on mixed-architecture fusion.
  arXiv  Detail & Related papers  (2025-04-15T08:19:12Z)
• Connecting the geometry and dynamics of many-body complex systems with message passing neural operators [1.8434042562191815]
  We introduce a scalable AI framework, ROMA, for learning multiscale evolution operators of many-body complex systems.<n>An attention mechanism is used to model multiscale interactions by connecting geometric representations of local subgraphs and dynamical operators.<n>We demonstrate that the ROMA framework improves scalability and positive transfer between forecasting and effective dynamics tasks.
  arXiv  Detail & Related papers  (2025-02-21T20:04:09Z)
• Orchid: Flexible and Data-Dependent Convolution for Sequence Modeling [4.190836962132713]
  This paper introduces Orchid, a novel architecture designed to address the quadratic complexity of traditional attention mechanisms. At the core of this architecture lies a new data-dependent global convolution layer, which contextually adapts its conditioned kernel on input sequence. We evaluate the proposed model across multiple domains, including language modeling and image classification, to highlight its performance and generality.
  arXiv  Detail & Related papers  (2024-02-28T17:36:45Z)
• Autoencoders for discovering manifold dimension and coordinates in data from complex dynamical systems [0.0]
  Autoencoder framework combines implicit regularization with internal linear layers and $L$ regularization (weight decay) We show that this framework can be naturally extended for applications of state-space modeling and forecasting.
  arXiv  Detail & Related papers  (2023-05-01T21:14:47Z)
• PDSketch: Integrated Planning Domain Programming and Learning [86.07442931141637]
  We present a new domain definition language, named PDSketch. It allows users to flexibly define high-level structures in the transition models. Details of the transition model will be filled in by trainable neural networks.
  arXiv  Detail & Related papers  (2023-03-09T18:54:12Z)
• Multi-Scale Semantics-Guided Neural Networks for Efficient Skeleton-Based Human Action Recognition [140.18376685167857]
  A simple yet effective multi-scale semantics-guided neural network is proposed for skeleton-based action recognition. MS-SGN achieves the state-of-the-art performance on the NTU60, NTU120, and SYSU datasets.
  arXiv  Detail & Related papers  (2021-11-07T03:50:50Z)
• XY Neural Networks [0.0]
  We show how to build complex structures for machine learning based on the XY model's nonlinear blocks. The final target is to reproduce the deep learning architectures, which can perform complicated tasks.
  arXiv  Detail & Related papers  (2021-03-31T17:47:10Z)
• Dual-constrained Deep Semi-Supervised Coupled Factorization Network with Enriched Prior [80.5637175255349]
  We propose a new enriched prior based Dual-constrained Deep Semi-Supervised Coupled Factorization Network, called DS2CF-Net. To ex-tract hidden deep features, DS2CF-Net is modeled as a deep-structure and geometrical structure-constrained neural network. Our network can obtain state-of-the-art performance for representation learning and clustering.
  arXiv  Detail & Related papers  (2020-09-08T13:10:21Z)
• S2RMs: Spatially Structured Recurrent Modules [105.0377129434636]
  We take a step towards exploiting dynamic structure that are capable of simultaneously exploiting both modular andtemporal structures. We find our models to be robust to the number of available views and better capable of generalization to novel tasks without additional training.
  arXiv  Detail & Related papers  (2020-07-13T17:44:30Z)
• Self-organization of multi-layer spiking neural networks [4.859525864236446]
  A key mechanism that enables the formation of complex architecture in the developing brain is the emergence of traveling-temporal waves of neuronal activity. We propose a modular tool-kit in the form of a dynamical system that can be seamlessly stacked to assemble multi-layer neural networks. Our framework leads to the self-organization of a wide variety of architectures, ranging from multi-layer perceptrons to autoencoders.
  arXiv  Detail & Related papers  (2020-06-12T01:44:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.