Informed Equation Learning

• URL: http://arxiv.org/abs/2105.06331v1
• Date: Thu, 13 May 2021 14:37:25 GMT
• Title: Informed Equation Learning
• Abstract summary: We propose an informed equation learning system for science and engineering. It incorporates expert knowledge about what are permitted or prohibited equation components. Our system then utilizes a robust method to learn equations with atomic functions exhibiting singularities.
• Score: 31.29253826586943
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Distilling data into compact and interpretable analytic equations is one of the goals of science. Instead, contemporary supervised machine learning methods mostly produce unstructured and dense maps from input to output. Particularly in deep learning, this property is owed to the generic nature of simple standard link functions. To learn equations rather than maps, standard non-linearities can be replaced with structured building blocks of atomic functions. However, without strong priors on sparsity and structure, representational complexity and numerical conditioning limit this direct approach. To scale to realistic settings in science and engineering, we propose an informed equation learning system. It provides a way to incorporate expert knowledge about what are permitted or prohibited equation components, as well as a domain-dependent structured sparsity prior. Our system then utilizes a robust method to learn equations with atomic functions exhibiting singularities, as e.g. logarithm and division. We demonstrate several artificial and real-world experiments from the engineering domain, in which our system learns interpretable models of high predictive power.

Related papers

• Symbolic Regression via Neural Networks [0.0]
  We present a novel approach that combines the flexibility and accuracy of deep learning approaches with the utility of symbolic solutions.<n>We first describe the architecture for our model, then show the accuracy of our algorithm across a range of classical dynamical systems.
  arXiv  Detail & Related papers  (2026-05-05T22:43:35Z)
• Hierarchical Physics-Embedded Learning for Spatiotemporal Dynamical Systems [12.832325257647128]
  We propose a hierarchical physics-embedded learning framework that advances both forwardtemporal prediction and inverse discovery of physical laws.<n>Known physical laws are directly embedded into the models computational graph, guaranteeing physical consistency.<n>By building the framework upon adaptive Neural Operators, we can effectively capture the non-local dependencies and high-order operators characteristic of dynamical systems.
  arXiv  Detail & Related papers  (2025-10-29T09:18:41Z)
• Learning Causal Structure Distributions for Robust Planning [53.753366558072806]
  We find that learning the functional relationships while accounting for the uncertainty about the structural information leads to more robust dynamics models.<n>This in contrast with common model-learning methods that ignore the causal structure and fail to leverage the sparsity of interactions in robotic systems.<n>We show that our model can be used to learn the dynamics of a robot, which together with a sampling-based planner can be used to perform new tasks in novel environments.
  arXiv  Detail & Related papers  (2025-08-08T22:43:17Z)
• Physics-informed features in supervised machine learning [0.0]
  Supervised machine learning involves approximating an unknown functional relationship from a limited dataset of features and corresponding labels. This study proposes a physics-informed approach to feature-based machine learning that constructs non-linear feature maps informed by physical laws and dimensional analysis.
  arXiv  Detail & Related papers  (2025-04-23T21:45:49Z)
• DUE: A Deep Learning Framework and Library for Modeling Unknown Equations [2.832405269873282]
  Deep Unknown Equations (DUE) is an open-source software package designed to facilitate the data-driven modeling of unknown equations. We introduce a systematic framework for data-driven modeling of unknown equations using deep learning. DUE serves as an educational tool for classroom instruction, enabling students and newcomers to gain hands-on experience with differential equations.
  arXiv  Detail & Related papers  (2025-04-14T16:20:55Z)
• Algebraic Machine Learning: Learning as computing an algebraic decomposition of a task [41.94295877935867]
  We propose an alternative foundation based on Abstract Algebra, with mathematics that facilitates the analysis of learning. In this approach, the goal of the task and the data are encoded as axioms of an algebra, and a model is obtained where only these axioms and their logical consequences hold. We validate this new learning principle on standard datasets such as MNIST, FashionMNIST, CIFAR-10, and medical images, achieving performance comparable to optimized multilayer perceptrons.
  arXiv  Detail & Related papers  (2025-02-27T10:13:42Z)
• No Equations Needed: Learning System Dynamics Without Relying on Closed-Form ODEs [56.78271181959529]
  This paper proposes a conceptual shift to modeling low-dimensional dynamical systems by departing from the traditional two-step modeling process. Instead of first discovering a closed-form equation and then analyzing it, our approach, direct semantic modeling, predicts the semantic representation of the dynamical system. Our approach not only simplifies the modeling pipeline but also enhances the transparency and flexibility of the resulting models.
  arXiv  Detail & Related papers  (2025-01-30T18:36:48Z)
• Knowledge-aware equation discovery with automated background knowledge extraction [50.79602839359522]
  We describe an algorithm that allows the discovery of unknown equations using automatically or manually extracted background knowledge. In this way, we mimic expertly chosen terms while preserving the possibility of obtaining any equation form. The paper shows that the extraction and use of knowledge allows it to outperform the SINDy algorithm in terms of search stability and robustness.
  arXiv  Detail & Related papers  (2024-12-31T13:51:31Z)
• Data Science Principles for Interpretable and Explainable AI [0.7581664835990121]
  Interpretable and interactive machine learning aims to make complex models more transparent and controllable. This review synthesizes key principles from the growing literature in this field.
  arXiv  Detail & Related papers  (2024-05-17T05:32:27Z)
• Large Language Models for Scientific Synthesis, Inference and Explanation [56.41963802804953]
  We show how large language models can perform scientific synthesis, inference, and explanation. We show that the large language model can augment this "knowledge" by synthesizing from the scientific literature. This approach has the further advantage that the large language model can explain the machine learning system's predictions.
  arXiv  Detail & Related papers  (2023-10-12T02:17:59Z)
• Discovering Interpretable Physical Models using Symbolic Regression and Discrete Exterior Calculus [55.2480439325792]
  We propose a framework that combines Symbolic Regression (SR) and Discrete Exterior Calculus (DEC) for the automated discovery of physical models. DEC provides building blocks for the discrete analogue of field theories, which are beyond the state-of-the-art applications of SR to physical problems. We prove the effectiveness of our methodology by re-discovering three models of Continuum Physics from synthetic experimental data.
  arXiv  Detail & Related papers  (2023-10-10T13:23:05Z)
• Towards true discovery of the differential equations [57.089645396998506]
  Differential equation discovery is a machine learning subfield used to develop interpretable models. This paper explores the prerequisites and tools for independent equation discovery without expert input.
  arXiv  Detail & Related papers  (2023-08-09T12:03:12Z)
• Neural Integral Equations [3.087238735145305]
  We introduce a method for learning unknown integral operators from data using an IE solver. We also present Attentional Neural Integral Equations (ANIE), which replaces the integral with self-attention.
  arXiv  Detail & Related papers  (2022-09-30T02:32:17Z)
• D-CIPHER: Discovery of Closed-form Partial Differential Equations [80.46395274587098]
  We propose D-CIPHER, which is robust to measurement artifacts and can uncover a new and very general class of differential equations. We further design a novel optimization procedure, CoLLie, to help D-CIPHER search through this class efficiently.
  arXiv  Detail & Related papers  (2022-06-21T17:59:20Z)
• NOMAD: Nonlinear Manifold Decoders for Operator Learning [17.812064311297117]
  Supervised learning in function spaces is an emerging area of machine learning research. We show NOMAD, a novel operator learning framework with a nonlinear decoder map capable of learning finite dimensional representations of nonlinear submanifolds in function spaces.
  arXiv  Detail & Related papers  (2022-06-07T19:52:44Z)
• Bayesian Learning to Discover Mathematical Operations in Governing Equations of Dynamic Systems [3.1544304017740634]
  This work presents a new representation for governing equations by designing the Mathematical Operation Network (MathONet) with a deep neural network-like hierarchical structure. An MathONet is typically regarded as a super-graph with a redundant structure, a sub-graph of which can yield the governing equation.
  arXiv  Detail & Related papers  (2022-06-01T10:31:14Z)
• Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning [93.18238573921629]
  We study how Ensemble of deep learning models can improve test accuracy, and how the superior performance of ensemble can be distilled into a single model. We show that ensemble/knowledge distillation in deep learning works very differently from traditional learning theory. We prove that self-distillation can also be viewed as implicitly combining ensemble and knowledge distillation to improve test accuracy.
  arXiv  Detail & Related papers  (2020-12-17T18:34:45Z)
• Model-Based Deep Learning [155.063817656602]
  Signal processing, communications, and control have traditionally relied on classical statistical modeling techniques. Deep neural networks (DNNs) use generic architectures which learn to operate from data, and demonstrate excellent performance. We are interested in hybrid techniques that combine principled mathematical models with data-driven systems to benefit from the advantages of both approaches.
  arXiv  Detail & Related papers  (2020-12-15T16:29:49Z)
• Learning ODE Models with Qualitative Structure Using Gaussian Processes [0.6882042556551611]
  In many contexts explicit data collection is expensive and learning algorithms must be data-efficient to be feasible. We propose an approach to learning a vector field of differential equations using sparse Gaussian Processes. We show that this combination improves extrapolation performance and long-term behaviour significantly, while also reducing the computational cost.
  arXiv  Detail & Related papers  (2020-11-10T19:34:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.