MEP-Net: Generating Solutions to Scientific Problems with Limited Knowledge by Maximum Entropy Principle

• URL: http://arxiv.org/abs/2412.02090v1
• Date: Tue, 03 Dec 2024 02:21:29 GMT
• Title: MEP-Net: Generating Solutions to Scientific Problems with Limited Knowledge by Maximum Entropy Principle
• Authors: Wuyue Yang, Liangrong Peng, Guojie Li, Liu Hong,
• Abstract summary: This paper proposes a novel neural network architecture, the MEP-Net, which combines the MEP with neural networks to generate probability distributions from moment constraints.<n>We show that the MEP-Net can be particularly useful in modeling the evolution of probability distributions in biochemical reaction networks and in generating complex distributions from data.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Maximum entropy principle (MEP) offers an effective and unbiased approach to inferring unknown probability distributions when faced with incomplete information, while neural networks provide the flexibility to learn complex distributions from data. This paper proposes a novel neural network architecture, the MEP-Net, which combines the MEP with neural networks to generate probability distributions from moment constraints. We also provide a comprehensive overview of the fundamentals of the maximum entropy principle, its mathematical formulations, and a rigorous justification for its applicability for non-equilibrium systems based on the large deviations principle. Through fruitful numerical experiments, we demonstrate that the MEP-Net can be particularly useful in modeling the evolution of probability distributions in biochemical reaction networks and in generating complex distributions from data.

Related papers

• Bayesian Reasoning Enabled by Spin-Orbit Torque Magnetic Tunnel Junctions [7.081096702778852]
  We present proof-of-concept experiments demonstrating the use of spin-orbit torque magnetic tunnel junctions (SOT-MTJs) in Bayesian network reasoning. The parameters of the network can also approach the optimum through a simple point-by-point training algorithm. We developed a simple medical diagnostic system using the SOT-MTJ as a random number generator and sampler.
  arXiv  Detail & Related papers  (2025-04-11T05:02:27Z)
• VAEs and GANs: Implicitly Approximating Complex Distributions with Simple Base Distributions and Deep Neural Networks -- Principles, Necessity, and Limitations [0.0]
  This tutorial focuses on the fundamental architectures of Variational Autoencoders (VAE) and Generative Adversarial Networks (GAN) VAE and GAN utilize simple distributions, such as Gaussians, as a basis and leverage the powerful nonlinear transformation capabilities of neural networks to approximate arbitrarily complex distributions.
  arXiv  Detail & Related papers  (2025-02-28T02:34:14Z)
• Probabilistic Neural Networks (PNNs) for Modeling Aleatoric Uncertainty in Scientific Machine Learning [2.348041867134616]
  This paper investigates the use of probabilistic neural networks (PNNs) to model aleatoric uncertainty. PNNs generate probability distributions for the target variable, allowing the determination of both predicted means and intervals in regression scenarios. In a real-world scientific machine learning context, PNNs yield remarkably accurate output mean estimates with R-squared scores approaching 0.97, and their predicted intervals exhibit a high correlation coefficient of nearly 0.80.
  arXiv  Detail & Related papers  (2024-02-21T17:15:47Z)
• Learning Theory of Distribution Regression with Neural Networks [6.961253535504979]
  We establish an approximation theory and a learning theory of distribution regression via a fully connected neural network (FNN) In contrast to the classical regression methods, the input variables of distribution regression are probability measures.
  arXiv  Detail & Related papers  (2023-07-07T09:49:11Z)
• PDE+: Enhancing Generalization via PDE with Adaptive Distributional Diffusion [66.95761172711073]
  generalization of neural networks is a central challenge in machine learning. We propose to enhance it directly through the underlying function of neural networks, rather than focusing on adjusting input data. We put this theoretical framework into practice as $textbfPDE+$ ($textbfPDE$ with $textbfA$daptive $textbfD$istributional $textbfD$iffusion)
  arXiv  Detail & Related papers  (2023-05-25T08:23:26Z)
• Stretched and measured neural predictions of complex network dynamics [2.1024950052120417]
  Data-driven approximations of differential equations present a promising alternative to traditional methods for uncovering a model of dynamical systems. A recently employed machine learning tool for studying dynamics is neural networks, which can be used for data-driven solution finding or discovery of differential equations. We show that extending the model's generalizability beyond traditional statistical learning theory limits is feasible.
  arXiv  Detail & Related papers  (2023-01-12T09:44:59Z)
• Physics Informed Neural Network for Dynamic Stress Prediction [10.588266927411434]
  A Physics Informed Neural Network (PINN) model is proposed to predict the entire sequence of stress distribution based on Finite Element simulations. Using automatic differentiation, we embed a PDE into a deep neural network's loss function to incorporate information from measurements and PDEs. The PINN-Stress model can predict the sequence of stress distribution in almost real-time and can generalize better than the model without PINN.
  arXiv  Detail & Related papers  (2022-11-28T16:03:21Z)
• FF-NSL: Feed-Forward Neural-Symbolic Learner [70.978007919101]
  This paper introduces a neural-symbolic learning framework, called Feed-Forward Neural-Symbolic Learner (FF-NSL) FF-NSL integrates state-of-the-art ILP systems based on the Answer Set semantics, with neural networks, in order to learn interpretable hypotheses from labelled unstructured data.
  arXiv  Detail & Related papers  (2021-06-24T15:38:34Z)
• Influence Estimation and Maximization via Neural Mean-Field Dynamics [60.91291234832546]
  We propose a novel learning framework using neural mean-field (NMF) dynamics for inference and estimation problems. Our framework can simultaneously learn the structure of the diffusion network and the evolution of node infection probabilities.
  arXiv  Detail & Related papers  (2021-06-03T00:02:05Z)
• On Energy-Based Models with Overparametrized Shallow Neural Networks [44.74000986284978]
  Energy-based models (EBMs) are a powerful framework for generative modeling. In this work we focus on shallow neural networks. We show that models trained in the so-called "active" regime provide a statistical advantage over their associated "lazy" or kernel regime.
  arXiv  Detail & Related papers  (2021-04-15T15:34:58Z)
• General stochastic separation theorems with optimal bounds [68.8204255655161]
  Phenomenon of separability was revealed and used in machine learning to correct errors of Artificial Intelligence (AI) systems and analyze AI instabilities. Errors or clusters of errors can be separated from the rest of the data. The ability to correct an AI system also opens up the possibility of an attack on it, and the high dimensionality induces vulnerabilities caused by the same separability.
  arXiv  Detail & Related papers  (2020-10-11T13:12:41Z)
• Resource Allocation via Graph Neural Networks in Free Space Optical Fronthaul Networks [119.81868223344173]
  This paper investigates the optimal resource allocation in free space optical (FSO) fronthaul networks. We consider the graph neural network (GNN) for the policy parameterization to exploit the FSO network structure. The primal-dual learning algorithm is developed to train the GNN in a model-free manner, where the knowledge of system models is not required.
  arXiv  Detail & Related papers  (2020-06-26T14:20:48Z)
• GINNs: Graph-Informed Neural Networks for Multiscale Physics [1.1470070927586016]
  Graph-Informed Neural Network (GINN) is a hybrid approach combining deep learning with probabilistic graphical models (PGMs) GINNs produce kernel density estimates of relevant non-Gaussian, skewed QoIs with tight confidence intervals.
  arXiv  Detail & Related papers  (2020-06-26T05:47:45Z)
• Network Diffusions via Neural Mean-Field Dynamics [52.091487866968286]
  We propose a novel learning framework for inference and estimation problems of diffusion on networks. Our framework is derived from the Mori-Zwanzig formalism to obtain an exact evolution of the node infection probabilities. Our approach is versatile and robust to variations of the underlying diffusion network models.
  arXiv  Detail & Related papers  (2020-06-16T18:45:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.