Fourier-Invertible Neural Encoder (FINE) for Homogeneous Flows

• URL: http://arxiv.org/abs/2505.15329v2
• Date: Sat, 14 Jun 2025 06:53:29 GMT
• Title: Fourier-Invertible Neural Encoder (FINE) for Homogeneous Flows
• Authors: Anqiao Ouyang, Hongyi Ke, Qi Wang,
• Abstract summary: Invertible neural networks have attracted attention for their compactness, interpretability, and information-preserving properties.<n>We propose the Fourier-Invertible Neural (FINE), which combines invertible monotonic activation functions with reversible filter structures.
• Score: 4.095418032380801
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Invertible neural architectures have recently attracted attention for their compactness, interpretability, and information-preserving properties. In this work, we propose the Fourier-Invertible Neural Encoder (FINE), which combines invertible monotonic activation functions with reversible filter structures, and could be extended using Invertible ResNets. This architecture is examined in learning low-dimensional representations of one-dimensional nonlinear wave interactions and exact circular translation symmetry. Dimensionality is preserved across layers, except for a Fourier truncation step in the latent space, which enables dimensionality reduction while maintaining shift equivariance and interpretability. Our results demonstrate that FINE significantly outperforms classical linear methods such as Discrete Fourier Transformation (DFT) and Proper Orthogonal Decomposition (POD), and achieves reconstruction accuracy better than conventional deep autoencoders with convolutional layers (CNN) - while using substantially smaller models and offering superior physical interpretability. These findings suggest that invertible single-neuron networks, when combined with spectral truncation, offer a promising framework for learning compact and interpretable representations of physics datasets, and symmetry-aware representation learning in physics-informed machine learning.

Related papers

• Generalized Linear Mode Connectivity for Transformers [87.32299363530996]
  A striking phenomenon is linear mode connectivity (LMC), where independently trained models can be connected by low- or zero-loss paths.<n>Prior work has predominantly focused on neuron re-ordering through permutations, but such approaches are limited in scope.<n>We introduce a unified framework that captures four symmetry classes: permutations, semi-permutations, transformations, and general invertible maps.<n>This generalization enables, for the first time, the discovery of low- and zero-barrier linear paths between independently trained Vision Transformers and GPT-2 models.
  arXiv  Detail & Related papers  (2025-06-28T01:46:36Z)
• Geometry aware inference of steady state PDEs using Equivariant Neural Fields representations [0.0]
  We introduce enf2enf, an encoder--decoder methodology for predicting steady-state Partial Differential Equations.<n>Our method supports real time inference and zero-shot super-resolution, enabling efficient training on low-resolution meshes.
  arXiv  Detail & Related papers  (2025-04-24T08:30:32Z)
• OTLRM: Orthogonal Learning-based Low-Rank Metric for Multi-Dimensional Inverse Problems [14.893020063373022]
  We introduce a novel data-driven generative low-rank t-SVD model based on the learnable orthogonal transform.<n>We also propose a low-rank solver as a generalization of SVT, which utilizes an efficient representation of generative networks to obtain low-rank structures.
  arXiv  Detail & Related papers  (2024-12-15T12:28:57Z)
• DimOL: Dimensional Awareness as A New 'Dimension' in Operator Learning [60.58067866537143]
  We introduce DimOL (Dimension-aware Operator Learning), drawing insights from dimensional analysis.<n>To implement DimOL, we propose the ProdLayer, which can be seamlessly integrated into FNO-based and Transformer-based PDE solvers.<n> Empirically, DimOL models achieve up to 48% performance gain within the PDE datasets.
  arXiv  Detail & Related papers  (2024-10-08T10:48:50Z)
• Physics-embedded Fourier Neural Network for Partial Differential Equations [35.41134465442465]
  We introduce Physics-embedded Fourier Neural Networks (PeFNN) with flexible and explainable error. PeFNN is designed to enforce momentum conservation and yields interpretable nonlinear expressions. We demonstrate its outstanding performance for challenging real-world applications such as large-scale flood simulations.
  arXiv  Detail & Related papers  (2024-07-15T18:30:39Z)
• Domain Agnostic Fourier Neural Operators [15.29112632863168]
  We introduce domain agnostic Fourier neural operator (DAFNO) for learning surrogates with irregular geometries and evolving domains. The key idea is to incorporate a smoothed characteristic function in the integral layer architecture of FNOs. DAFNO has achieved state-of-the-art accuracy as compared to baseline neural operator models.
  arXiv  Detail & Related papers  (2023-04-30T13:29:06Z)
• VTAE: Variational Transformer Autoencoder with Manifolds Learning [144.0546653941249]
  Deep generative models have demonstrated successful applications in learning non-linear data distributions through a number of latent variables. The nonlinearity of the generator implies that the latent space shows an unsatisfactory projection of the data space, which results in poor representation learning. We show that geodesics and accurate computation can substantially improve the performance of deep generative models.
  arXiv  Detail & Related papers  (2023-04-03T13:13:19Z)
• Adaptive Fourier Neural Operators: Efficient Token Mixers for Transformers [55.90468016961356]
  We propose an efficient token mixer that learns to mix in the Fourier domain. AFNO is based on a principled foundation of operator learning. It can handle a sequence size of 65k and outperforms other efficient self-attention mechanisms.
  arXiv  Detail & Related papers  (2021-11-24T05:44:31Z)
• Convolutional Filtering and Neural Networks with Non Commutative Algebras [153.20329791008095]
  We study the generalization of non commutative convolutional neural networks. We show that non commutative convolutional architectures can be stable to deformations on the space of operators.
  arXiv  Detail & Related papers  (2021-08-23T04:22:58Z)
• Adaptive Latent Space Tuning for Non-Stationary Distributions [62.997667081978825]
  We present a method for adaptive tuning of the low-dimensional latent space of deep encoder-decoder style CNNs. We demonstrate our approach for predicting the properties of a time-varying charged particle beam in a particle accelerator.
  arXiv  Detail & Related papers  (2021-05-08T03:50:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.