PCENet: High Dimensional Surrogate Modeling for Learning Uncertainty

• URL: http://arxiv.org/abs/2202.05063v3
• Date: Mon, 04 Aug 2025 20:32:14 GMT
• Title: PCENet: High Dimensional Surrogate Modeling for Learning Uncertainty
• Authors: Paz Fink Shustin, Shashanka Ubaru, Małgorzata J. Zimoń, Songtao Lu, Vasileios Kalantzis, Lior Horesh, Haim Avron,
• Abstract summary: We introduce a dimensionality reduction surrogate modeling (DRSM) approach for representation learning and uncertainty quantification.<n>The approach involves a two-stage learning process: 1) employing a variational autoencoder to learn a low-dimensional representation of the input data distribution; and 2) harnessing the chaos expansion (PCE) to map the low dimensional distribution to the output target.
• Score: 33.334475200283435
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Learning data representations under uncertainty is an important task that emerges in numerous scientific computing and data analysis applications. However, uncertainty quantification techniques are computationally intensive and become prohibitively expensive for high-dimensional data. In this study, we introduce a dimensionality reduction surrogate modeling (DRSM) approach for representation learning and uncertainty quantification that aims to deal with data of moderate to high dimensions. The approach involves a two-stage learning process: 1) employing a variational autoencoder to learn a low-dimensional representation of the input data distribution; and 2) harnessing polynomial chaos expansion (PCE) formulation to map the low dimensional distribution to the output target. The model enables us to (a) capture the system dynamics efficiently in the low-dimensional latent space, (b) learn under uncertainty, a representation of the data and a mapping between input and output distributions, (c) estimate this uncertainty in the high-dimensional data system, and (d) match high-order moments of the output distribution; without any prior statistical assumptions on the data. Numerical results are presented to illustrate the performance of the proposed method.

Related papers

• INFO-SEDD: Continuous Time Markov Chains as Scalable Information Metrics Estimators [7.399561232927219]
  INFO-SEDD is a novel method for estimating information-theoretic quantities of discrete data, including mutual information and entropy. Our approach requires the training of a single parametric model, offering significant computational and memory advantages. Our experiments demonstrate that INFO-SEDD is robust and outperforms neural competitors that rely on embedding techniques.
  arXiv  Detail & Related papers  (2025-02-26T14:40:00Z)
• Bayesian Estimation and Tuning-Free Rank Detection for Probability Mass Function Tensors [17.640500920466984]
  This paper presents a novel framework for estimating the joint PMF and automatically inferring its rank from observed data. We derive a deterministic solution based on variational inference (VI) to approximate the posterior distributions of various model parameters. Additionally, we develop a scalable version of the VI-based approach by leveraging variational inference (SVI) Experiments involving both synthetic data and real movie recommendation data illustrate the advantages of our VI and SVI-based methods in terms of estimation accuracy, automatic rank detection, and computational efficiency.
  arXiv  Detail & Related papers  (2024-10-08T20:07:49Z)
• Learning Latent Graph Structures and their Uncertainty [63.95971478893842]
  Graph Neural Networks (GNNs) use relational information as an inductive bias to enhance the model's accuracy. As task-relevant relations might be unknown, graph structure learning approaches have been proposed to learn them while solving the downstream prediction task.
  arXiv  Detail & Related papers  (2024-05-30T10:49:22Z)
• Proximal Symmetric Non-negative Latent Factor Analysis: A Novel Approach to Highly-Accurate Representation of Undirected Weighted Networks [2.1797442801107056]
  Undirected Weighted Network (UWN) is commonly found in big data-related applications. Existing models fail in either modeling its intrinsic symmetry or low-data density. Proximal Symmetric Nonnegative Latent-factor-analysis model is proposed.
  arXiv  Detail & Related papers  (2023-06-06T13:03:24Z)
• Application of probabilistic modeling and automated machine learning framework for high-dimensional stress field [1.073039474000799]
  We propose an end-to-end approach that maps a high-dimensional image like input to an output of high dimensionality or its key statistics. Our approach uses two main framework that perform three steps: a) reduce the input and output from a high-dimensional space to a reduced or low-dimensional space, b) model the input-output relationship in the low-dimensional space, and c) enable the incorporation of domain-specific physical constraints as masks.
  arXiv  Detail & Related papers  (2023-03-15T13:10:58Z)
• IB-UQ: Information bottleneck based uncertainty quantification for neural function regression and neural operator learning [11.5992081385106]
  We propose a novel framework for uncertainty quantification via information bottleneck (IB-UQ) for scientific machine learning tasks. We incorporate the bottleneck by a confidence-aware encoder, which encodes inputs into latent representations according to the confidence of the input data. We also propose a data augmentation based information bottleneck objective which can enhance the quality of the extrapolation uncertainty.
  arXiv  Detail & Related papers  (2023-02-07T05:56:42Z)
• Extension of Dynamic Mode Decomposition for dynamic systems with incomplete information based on t-model of optimal prediction [69.81996031777717]
  The Dynamic Mode Decomposition has proved to be a very efficient technique to study dynamic data. The application of this approach becomes problematic if the available data is incomplete because some dimensions of smaller scale either missing or unmeasured. We consider a first-order approximation of the Mori-Zwanzig decomposition, state the corresponding optimization problem and solve it with the gradient-based optimization method.
  arXiv  Detail & Related papers  (2022-02-23T11:23:59Z)
• Graph Embedding via High Dimensional Model Representation for Hyperspectral Images [9.228929858529678]
  Learning the manifold structure of remote sensing images is of paramount relevance for modeling and understanding processes. Manor learning methods have shown excellent performance to deal with hyperspectral image (HSI) analysis. A common assumption to deal with the problem is that the transformation between the high-dimensional input space and the (typically low) latent space is linear. The proposed method is compared to manifold learning methods along with its linear counterparts and achieves promising performance in terms of classification accuracy of a representative set of hyperspectral images.
  arXiv  Detail & Related papers  (2021-11-29T16:42:15Z)
• Data-driven Uncertainty Quantification in Computational Human Head Models [0.6745502291821954]
  Modern biofidelic head model simulations are associated with very high computational cost and high-dimensional inputs and outputs. In this study, a two-stage, data-driven manifold learning-based framework is proposed for uncertainty quantification (UQ) of computational head models. It is demonstrated that the surrogate models provide highly accurate approximations of the computational model while significantly reducing the computational cost.
  arXiv  Detail & Related papers  (2021-10-29T05:42:31Z)
• Low-rank statistical finite elements for scalable model-data synthesis [0.8602553195689513]
  statFEM acknowledges a priori model misspecification, by embedding forcing within the governing equations. The method reconstructs the observed data-generating processes with minimal loss of information. This article overcomes this hurdle by embedding a low-rank approximation of the underlying dense covariance matrix.
  arXiv  Detail & Related papers  (2021-09-10T09:51:43Z)
• Efficient training of lightweight neural networks using Online Self-Acquired Knowledge Distillation [51.66271681532262]
  Online Self-Acquired Knowledge Distillation (OSAKD) is proposed, aiming to improve the performance of any deep neural model in an online manner. We utilize k-nn non-parametric density estimation technique for estimating the unknown probability distributions of the data samples in the output feature space.
  arXiv  Detail & Related papers  (2021-08-26T14:01:04Z)
• Incorporating Causal Graphical Prior Knowledge into Predictive Modeling via Simple Data Augmentation [92.96204497841032]
  Causal graphs (CGs) are compact representations of the knowledge of the data generating processes behind the data distributions. We propose a model-agnostic data augmentation method that allows us to exploit the prior knowledge of the conditional independence (CI) relations. We experimentally show that the proposed method is effective in improving the prediction accuracy, especially in the small-data regime.
  arXiv  Detail & Related papers  (2021-02-27T06:13:59Z)
• Graph Embedding with Data Uncertainty [113.39838145450007]
  spectral-based subspace learning is a common data preprocessing step in many machine learning pipelines. Most subspace learning methods do not take into consideration possible measurement inaccuracies or artifacts that can lead to data with high uncertainty.
  arXiv  Detail & Related papers  (2020-09-01T15:08:23Z)
• Learning while Respecting Privacy and Robustness to Distributional Uncertainties and Adversarial Data [66.78671826743884]
  The distributionally robust optimization framework is considered for training a parametric model. The objective is to endow the trained model with robustness against adversarially manipulated input data. Proposed algorithms offer robustness with little overhead.
  arXiv  Detail & Related papers  (2020-07-07T18:25:25Z)
• Deep Dimension Reduction for Supervised Representation Learning [51.10448064423656]
  We propose a deep dimension reduction approach to learning representations with essential characteristics. The proposed approach is a nonparametric generalization of the sufficient dimension reduction method. We show that the estimated deep nonparametric representation is consistent in the sense that its excess risk converges to zero.
  arXiv  Detail & Related papers  (2020-06-10T14:47:43Z)
• Two-Dimensional Semi-Nonnegative Matrix Factorization for Clustering [50.43424130281065]
  We propose a new Semi-Nonnegative Matrix Factorization method for 2-dimensional (2D) data, named TS-NMF. It overcomes the drawback of existing methods that seriously damage the spatial information of the data by converting 2D data to vectors in a preprocessing step.
  arXiv  Detail & Related papers  (2020-05-19T05:54:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.