Formalizing the presumption of independence

• URL: http://arxiv.org/abs/2211.06738v1
• Date: Sat, 12 Nov 2022 20:28:19 GMT
• Title: Formalizing the presumption of independence
• Authors: Paul Christiano, Eric Neyman, Mark Xu
• Abstract summary: A key ingredient in such reasoning is the use of a "default" estimate of $mathbbE[XY] = mathbbE[X] mathbbE[Y]$. Reasoning based on this is commonplace, intuitively compelling, and often quite successful -- but completely informal. We present our main open problem: is there an estimator that formalizes intuitively valid applications of the presumption of independence without also accepting spurious arguments?
• Score: 2.658812114255374
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Mathematical proof aims to deliver confident conclusions, but a very similar process of deduction can be used to make uncertain estimates that are open to revision. A key ingredient in such reasoning is the use of a "default" estimate of $\mathbb{E}[XY] = \mathbb{E}[X] \mathbb{E}[Y]$ in the absence of any specific information about the correlation between $X$ and $Y$, which we call *the presumption of independence*. Reasoning based on this heuristic is commonplace, intuitively compelling, and often quite successful -- but completely informal. In this paper we introduce the concept of a heuristic estimator as a potential formalization of this type of defeasible reasoning. We introduce a set of intuitively desirable coherence properties for heuristic estimators that are not satisfied by any existing candidates. Then we present our main open problem: is there a heuristic estimator that formalizes intuitively valid applications of the presumption of independence without also accepting spurious arguments?

Related papers

• Towards a Law of Iterated Expectations for Heuristic Estimators [10.287238241371922]
  In this work, we argue for the informal principle that a estimator ought not to be able to predict its own errors. We argue that an ideal estimator ought to satisfy two stronger properties in this vein. As an alternative approach, we explore *accuracy*: a property that states that $mathbbG(Y mid pi)$ has zero average error over a distribution of mathematical expressions.
  arXiv  Detail & Related papers  (2024-10-02T07:33:27Z)
• The Foundations of Tokenization: Statistical and Computational Concerns [51.370165245628975]
  Tokenization is a critical step in the NLP pipeline. Despite its recognized importance as a standard representation method in NLP, the theoretical underpinnings of tokenization are not yet fully understood. The present paper contributes to addressing this theoretical gap by proposing a unified formal framework for representing and analyzing tokenizer models.
  arXiv  Detail & Related papers  (2024-07-16T11:12:28Z)
• Betting on what is neither verifiable nor falsifiable [18.688474183114085]
  We propose an approach to betting on such events via options, or equivalently as bets on the outcome of a "verification-falsification game" Our work thus acts as an alternative to the existing framework of Garrabrant induction for logical uncertainty, and relates to the stance known as constructivism in the philosophy of mathematics.
  arXiv  Detail & Related papers  (2024-01-29T17:30:34Z)
• Uncovering Meanings of Embeddings via Partial Orthogonality [29.190972879474526]
  Machine learning tools often rely on embedding text as vectors of real numbers. We study how the semantic structure of language is encoded in the algebraic structure of such embeddings.
  arXiv  Detail & Related papers  (2023-10-26T17:34:32Z)
• Spectral theorem for dummies: A pedagogical discussion on quantum probability and random variable theory [0.0]
  John von Neumann's spectral theorem for self-adjoint operators is a cornerstone of quantum mechanics. This paper presents a plain-spoken formulation of this theorem in terms of Dirac's bra and ket notation.
  arXiv  Detail & Related papers  (2022-11-23T07:05:47Z)
• Checking Trustworthiness of Probabilistic Computations in a Typed Natural Deduction System [0.0]
  Derivability in TPTND is interpreted as the process of extracting $n$ samples with a certain frequency from a given categorical distribution. We present a computational semantics for the terms over which we reason and then the semantics of TPTND. We illustrate structural and metatheoretical properties, with particular focus on the ability to establish under which term evolutions and logical rules applications the notion of trustworhtiness can be preserved.
  arXiv  Detail & Related papers  (2022-06-26T17:55:32Z)
• Logical Satisfiability of Counterfactuals for Faithful Explanations in NLI [60.142926537264714]
  We introduce the methodology of Faithfulness-through-Counterfactuals. It generates a counterfactual hypothesis based on the logical predicates expressed in the explanation. It then evaluates if the model's prediction on the counterfactual is consistent with that expressed logic.
  arXiv  Detail & Related papers  (2022-05-25T03:40:59Z)
• Nonparametric Conditional Local Independence Testing [69.31200003384122]
  Conditional local independence is an independence relation among continuous time processes. No nonparametric test of conditional local independence has been available. We propose such a nonparametric test based on double machine learning.
  arXiv  Detail & Related papers  (2022-03-25T10:31:02Z)
• Causal Expectation-Maximisation [70.45873402967297]
  We show that causal inference is NP-hard even in models characterised by polytree-shaped graphs. We introduce the causal EM algorithm to reconstruct the uncertainty about the latent variables from data about categorical manifest variables. We argue that there appears to be an unnoticed limitation to the trending idea that counterfactual bounds can often be computed without knowledge of the structural equations.
  arXiv  Detail & Related papers  (2020-11-04T10:25:13Z)
• A Weaker Faithfulness Assumption based on Triple Interactions [89.59955143854556]
  We propose a weaker assumption that we call $2$-adjacency faithfulness. We propose a sound orientation rule for causal discovery that applies under weaker assumptions.
  arXiv  Detail & Related papers  (2020-10-27T13:04:08Z)
• Nonparametric Score Estimators [49.42469547970041]
  Estimating the score from a set of samples generated by an unknown distribution is a fundamental task in inference and learning of probabilistic models. We provide a unifying view of these estimators under the framework of regularized nonparametric regression. We propose score estimators based on iterative regularization that enjoy computational benefits from curl-free kernels and fast convergence.
  arXiv  Detail & Related papers  (2020-05-20T15:01:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.