Hyper-Reduced Autoencoders for Efficient and Accurate Nonlinear Model Reductions

• URL: http://arxiv.org/abs/2303.09630v1
• Date: Thu, 16 Mar 2023 20:18:33 GMT
• Title: Hyper-Reduced Autoencoders for Efficient and Accurate Nonlinear Model Reductions
• Authors: Jorio Cocola, John Tencer, Francesco Rizzi, Eric Parish, Patrick Blonigan
• Abstract summary: Projection-based model order reduction has been recently proposed for problems with slowly decaying Kolmogorov n-width. A disadvantage of the previously proposed methods is the potential high computational costs of training the networks on high-fidelity solution snapshots. We propose and analyze a novel method that overcomes this disadvantage by training a neural network only on subsampled versions of the high-fidelity solution snapshots.
• Score: 1.0499611180329804
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Projection-based model order reduction on nonlinear manifolds has been recently proposed for problems with slowly decaying Kolmogorov n-width such as advection-dominated ones. These methods often use neural networks for manifold learning and showcase improved accuracy over traditional linear subspace-reduced order models. A disadvantage of the previously proposed methods is the potential high computational costs of training the networks on high-fidelity solution snapshots. In this work, we propose and analyze a novel method that overcomes this disadvantage by training a neural network only on subsampled versions of the high-fidelity solution snapshots. This method coupled with collocation-based hyper-reduction and Gappy-POD allows for efficient and accurate surrogate models. We demonstrate the validity of our approach on a 2d Burgers problem.

Related papers

• Transfer Learning on Multi-Dimensional Data: A Novel Approach to Neural Network-Based Surrogate Modeling [0.0]
  Convolutional neural networks (CNNs) have gained popularity as the basis for such surrogate models. We propose training a CNN surrogate model on a mixture of numerical solutions to both the $d$-dimensional problem and its ($d-1$)-dimensional approximation. We demonstrate our approach on a multiphase flow test problem, using transfer learning to train a dense fully-convolutional encoder-decoder CNN on the two classes of data.
  arXiv  Detail & Related papers  (2024-10-16T05:07:48Z)
• Scalable Bayesian Inference in the Era of Deep Learning: From Gaussian Processes to Deep Neural Networks [0.5827521884806072]
  Large neural networks trained on large datasets have become the dominant paradigm in machine learning. This thesis develops scalable methods to equip neural networks with model uncertainty.
  arXiv  Detail & Related papers  (2024-04-29T23:38:58Z)
• The Convex Landscape of Neural Networks: Characterizing Global Optima and Stationary Points via Lasso Models [75.33431791218302]
  Deep Neural Network Network (DNN) models are used for programming purposes. In this paper we examine the use of convex neural recovery models. We show that all the stationary non-dimensional objective objective can be characterized as the standard a global subsampled convex solvers program. We also show that all the stationary non-dimensional objective objective can be characterized as the standard a global subsampled convex solvers program.
  arXiv  Detail & Related papers  (2023-12-19T23:04:56Z)
• An Optimization-based Deep Equilibrium Model for Hyperspectral Image Deconvolution with Convergence Guarantees [71.57324258813675]
  We propose a novel methodology for addressing the hyperspectral image deconvolution problem. A new optimization problem is formulated, leveraging a learnable regularizer in the form of a neural network. The derived iterative solver is then expressed as a fixed-point calculation problem within the Deep Equilibrium framework.
  arXiv  Detail & Related papers  (2023-06-10T08:25:16Z)
• Implicit Stochastic Gradient Descent for Training Physics-informed Neural Networks [51.92362217307946]
  Physics-informed neural networks (PINNs) have effectively been demonstrated in solving forward and inverse differential equation problems. PINNs are trapped in training failures when the target functions to be approximated exhibit high-frequency or multi-scale features. In this paper, we propose to employ implicit gradient descent (ISGD) method to train PINNs for improving the stability of training process.
  arXiv  Detail & Related papers  (2023-03-03T08:17:47Z)
• Regularizing Deep Neural Networks with Stochastic Estimators of Hessian Trace [1.933681537640272]
  We develop a novel regularization method for deep neural networks by penalizing the trace of Hessian. Experiments demonstrate that our method outperforms existing regularizers and data augmentation methods.
  arXiv  Detail & Related papers  (2022-08-11T16:51:27Z)
• Non-linear manifold ROM with Convolutional Autoencoders and Reduced Over-Collocation method [0.0]
  Non-affine parametric dependencies, nonlinearities and advection-dominated regimes of the model of interest can result in a slow Kolmogorov n-width decay. We implement the non-linear manifold method introduced by Carlberg et al [37] with hyper-reduction achieved through reduced over-collocation and teacher-student training of a reduced decoder. We test the methodology on a 2d non-linear conservation law and a 2d shallow water models, and compare the results obtained with a purely data-driven method for which the dynamics is evolved in time with a long-short term memory network
  arXiv  Detail & Related papers  (2022-03-01T11:16:50Z)
• Robust lEarned Shrinkage-Thresholding (REST): Robust unrolling for sparse recover [87.28082715343896]
  We consider deep neural networks for solving inverse problems that are robust to forward model mis-specifications. We design a new robust deep neural network architecture by applying algorithm unfolding techniques to a robust version of the underlying recovery problem. The proposed REST network is shown to outperform state-of-the-art model-based and data-driven algorithms in both compressive sensing and radar imaging problems.
  arXiv  Detail & Related papers  (2021-10-20T06:15:45Z)
• The Neural Network shifted-Proper Orthogonal Decomposition: a Machine Learning Approach for Non-linear Reduction of Hyperbolic Equations [0.0]
  In this work we approach the problem of automatically detecting the correct pre-processing transformation in a statistical learning framework. The purely data-driven method allowed us to generalise the existing approaches of linear subspace manipulation to non-linear hyperbolic problems with unknown advection fields. The proposed algorithm has been validated against simple test cases to benchmark its performances and later successfully applied to a multiphase simulation.
  arXiv  Detail & Related papers  (2021-08-14T15:13:35Z)
• Non-Gradient Manifold Neural Network [79.44066256794187]
  Deep neural network (DNN) generally takes thousands of iterations to optimize via gradient descent. We propose a novel manifold neural network based on non-gradient optimization.
  arXiv  Detail & Related papers  (2021-06-15T06:39:13Z)
• Belief Propagation Reloaded: Learning BP-Layers for Labeling Problems [83.98774574197613]
  We take one of the simplest inference methods, a truncated max-product Belief propagation, and add what is necessary to make it a proper component of a deep learning model. This BP-Layer can be used as the final or an intermediate block in convolutional neural networks (CNNs) The model is applicable to a range of dense prediction problems, is well-trainable and provides parameter-efficient and robust solutions in stereo, optical flow and semantic segmentation.
  arXiv  Detail & Related papers  (2020-03-13T13:11:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.