Priors for symbolic regression

• URL: http://arxiv.org/abs/2304.06333v2
• Date: Fri, 2 Jun 2023 07:50:25 GMT
• Title: Priors for symbolic regression
• Authors: Deaglan J. Bartlett, Harry Desmond, Pedro G. Ferreira
• Abstract summary: We develop methods to incorporate detailed prior information on both functions and their parameters into symbolic regression. Our prior on the structure of a function is based on a $n$-gram language model. We also develop a formalism based on the Fractional Bayes Factor to treat numerical parameter priors.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: When choosing between competing symbolic models for a data set, a human will naturally prefer the "simpler" expression or the one which more closely resembles equations previously seen in a similar context. This suggests a non-uniform prior on functions, which is, however, rarely considered within a symbolic regression (SR) framework. In this paper we develop methods to incorporate detailed prior information on both functions and their parameters into SR. Our prior on the structure of a function is based on a $n$-gram language model, which is sensitive to the arrangement of operators relative to one another in addition to the frequency of occurrence of each operator. We also develop a formalism based on the Fractional Bayes Factor to treat numerical parameter priors in such a way that models may be fairly compared though the Bayesian evidence, and explicitly compare Bayesian, Minimum Description Length and heuristic methods for model selection. We demonstrate the performance of our priors relative to literature standards on benchmarks and a real-world dataset from the field of cosmology.

Related papers

• Explaining Datasets in Words: Statistical Models with Natural Language Parameters [66.69456696878842]
  We introduce a family of statistical models -- including clustering, time series, and classification models -- parameterized by natural language predicates. We apply our framework to a wide range of problems: taxonomizing user chat dialogues, characterizing how they evolve across time, finding categories where one language model is better than the other.
  arXiv  Detail & Related papers  (2024-09-13T01:40:20Z)
• Diffusion posterior sampling for simulation-based inference in tall data settings [53.17563688225137]
  Simulation-based inference ( SBI) is capable of approximating the posterior distribution that relates input parameters to a given observation. In this work, we consider a tall data extension in which multiple observations are available to better infer the parameters of the model. We compare our method to recently proposed competing approaches on various numerical experiments and demonstrate its superiority in terms of numerical stability and computational cost.
  arXiv  Detail & Related papers  (2024-04-11T09:23:36Z)
• Latent Semantic Consensus For Deterministic Geometric Model Fitting [109.44565542031384]
  We propose an effective method called Latent Semantic Consensus (LSC) LSC formulates the model fitting problem into two latent semantic spaces based on data points and model hypotheses. LSC is able to provide consistent and reliable solutions within only a few milliseconds for general multi-structural model fitting.
  arXiv  Detail & Related papers  (2024-03-11T05:35:38Z)
• Learning to Select Prototypical Parts for Interpretable Sequential Data Modeling [7.376829794171344]
  We propose a Self-Explaining Selective Model (SESM) that uses a linear combination of prototypical concepts to explain its own predictions. For better interpretability, we design multiple constraints including diversity, stability, and locality as training objectives.
  arXiv  Detail & Related papers  (2022-12-07T01:42:47Z)
• Exhaustive Symbolic Regression [0.0]
  Exhaustive Symbolic Regression (ESR) is a rigorous method for combining preferences into a single objective. We apply it to a catalogue of cosmic chronometers and the Pantheon+ sample of supernovae to learn the Hubble rate. We make our code and full equation sets publicly available.
  arXiv  Detail & Related papers  (2022-11-21T13:48:52Z)
• Model-Based Counterfactual Synthesizer for Interpretation [40.01787107375103]
  We propose a Model-based Counterfactual Synthesizer (MCS) framework for interpreting machine learning models. We first analyze the model-based counterfactual process and construct a base synthesizer using a conditional generative adversarial net (CGAN) To better approximate the counterfactual universe for those rare queries, we novelly employ the umbrella sampling technique to conduct the MCS framework training.
  arXiv  Detail & Related papers  (2021-06-16T17:09:57Z)
• Autoregressive Score Matching [113.4502004812927]
  We propose autoregressive conditional score models (AR-CSM) where we parameterize the joint distribution in terms of the derivatives of univariable log-conditionals (scores) For AR-CSM models, this divergence between data and model distributions can be computed and optimized efficiently, requiring no expensive sampling or adversarial training. We show with extensive experimental results that it can be applied to density estimation on synthetic data, image generation, image denoising, and training latent variable models with implicit encoders.
  arXiv  Detail & Related papers  (2020-10-24T07:01:24Z)
• Predictive Complexity Priors [3.5547661483076998]
  We propose a functional prior that is defined by comparing the model's predictions to those of a reference model. Although originally defined on the model outputs, we transfer the prior to the model parameters via a change of variables. We apply our predictive complexity prior to high-dimensional regression, reasoning over neural network depth, and sharing of statistical strength for few-shot learning.
  arXiv  Detail & Related papers  (2020-06-18T18:39:49Z)
• Document Ranking with a Pretrained Sequence-to-Sequence Model [56.44269917346376]
  We show how a sequence-to-sequence model can be trained to generate relevance labels as "target words" Our approach significantly outperforms an encoder-only model in a data-poor regime.
  arXiv  Detail & Related papers  (2020-03-14T22:29:50Z)
• Pattern Similarity-based Machine Learning Methods for Mid-term Load Forecasting: A Comparative Study [0.0]
  We use pattern similarity-based methods for forecasting monthly electricity demand expressing annual seasonality. An integral part of the models is the time series representation using patterns of time series sequences. We consider four such models: nearest neighbor model, fuzzy neighborhood model, kernel regression model and general regression neural network.
  arXiv  Detail & Related papers  (2020-03-03T12:14:36Z)
• On the Discrepancy between Density Estimation and Sequence Generation [92.70116082182076]
  log-likelihood is highly correlated with BLEU when we consider models within the same family. We observe no correlation between rankings of models across different families.
  arXiv  Detail & Related papers  (2020-02-17T20:13:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.