An inferential measure of dependence between two systems using Bayesian model comparison

• URL: http://arxiv.org/abs/2412.06478v2
• Date: Tue, 10 Dec 2024 09:54:45 GMT
• Title: An inferential measure of dependence between two systems using Bayesian model comparison
• Authors: Guillaume Marrelec, Alain Giron,
• Abstract summary: dependence between $X$ and $Y$ in $D$ is quantified as $B(X,Y|D)$.<n>We discuss the consequences of using the Bayesian framework as well as the similarities and differences between $B(X,Y|D)$ and mutual information.
• Score: 3.683202928838613
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose to quantify dependence between two systems $X$ and $Y$ in a dataset $D$ based on the Bayesian comparison of two models: one, $H_0$, of statistical independence and another one, $H_1$, of dependence. In this framework, dependence between $X$ and $Y$ in $D$, denoted $B(X,Y|D)$, is quantified as $P(H_1|D)$, the posterior probability for the model of dependence given $D$, or any strictly increasing function thereof. It is therefore a measure of the evidence for dependence between $X$ and $Y$ as modeled by $H_1$ and observed in $D$. We review several statistical models and reconsider standard results in the light of $B(X,Y|D)$ as a measure of dependence. Using simulations, we focus on two specific issues: the effect of noise and the behavior of $B(X,Y|D)$ when $H_1$ has a parameter coding for the intensity of dependence. We then derive some general properties of $B(X,Y|D)$, showing that it quantifies the information contained in $D$ in favor of $H_1$ versus $H_0$. While some of these properties are typical of what is expected from a valid measure of dependence, others are novel and naturally appear as desired features for specific measures of dependence, which we call inferential. We finally put these results in perspective; in particular, we discuss the consequences of using the Bayesian framework as well as the similarities and differences between $B(X,Y|D)$ and mutual information.

Related papers

• Information-Computation Tradeoffs for Noiseless Linear Regression with Oblivious Contamination [65.37519531362157]
  We show that any efficient Statistical Query algorithm for this task requires VSTAT complexity at least $tildeOmega(d1/2/alpha2)$.
  arXiv  Detail & Related papers  (2025-10-12T15:42:44Z)
• Linear cost mutual information estimation and independence test of similar performance as HSIC [0.17478203318226307]
  HSIC (Hilbert-Schmidt Information Criterion)citeHSIC is considered the state-of-art method.<n>For size $n$ data sample it requires multiplication of $ntimes n$ matrices, what currently needs $sim O(n2.37)$ computational complexitycitemult, making it impractical for large data samples.<n>We discuss HCR as its linear cost practical alternative, in tests of even higher sensitivity to dependencies.
  arXiv  Detail & Related papers  (2025-08-25T13:45:25Z)
• An Interpretable Measure for Quantifying Predictive Dependence between Continuous Random Variables -- Extended Version [0.0]
  We introduce a novel measure that assesses the degree of association between continuous variables $X$ and $Y$. A key advantage of this measure is its interpretability. We evaluate the performance of our measure on over 90,000 real and synthetic datasets.
  arXiv  Detail & Related papers  (2025-01-18T16:25:20Z)
• Dimension-free Private Mean Estimation for Anisotropic Distributions [55.86374912608193]
  Previous private estimators on distributions over $mathRd suffer from a curse of dimensionality. We present an algorithm whose sample complexity has improved dependence on dimension.
  arXiv  Detail & Related papers  (2024-11-01T17:59:53Z)
• Statistical-Computational Trade-offs for Density Estimation [60.81548752871115]
  We show that for a broad class of data structures their bounds cannot be significantly improved. This is a novel emphstatistical-computational trade-off for density estimation.
  arXiv  Detail & Related papers  (2024-10-30T15:03:33Z)
• On Computing Pairwise Statistics with Local Differential Privacy [55.81991984375959]
  We study the problem of computing pairwise statistics, i.e., ones of the form $binomn2-1 sum_i ne j f(x_i, x_j)$, where $x_i$ denotes the input to the $i$th user, with differential privacy (DP) in the local model. This formulation captures important metrics such as Kendall's $tau$ coefficient, Area Under Curve, Gini's mean difference, Gini's entropy, etc.
  arXiv  Detail & Related papers  (2024-06-24T04:06:09Z)
• Data Structures for Density Estimation [66.36971978162461]
  Given a sublinear (in $n$) number of samples from $p$, our main result is the first data structure that identifies $v_i$ in time sublinear in $k$. We also give an improved version of the algorithm of Acharya et al. that reports $v_i$ in time linear in $k$.
  arXiv  Detail & Related papers  (2023-06-20T06:13:56Z)
• Online Learning with Adversaries: A Differential-Inclusion Analysis [52.43460995467893]
  We introduce an observation-matrix-based framework for fully asynchronous online Federated Learning with adversaries. Our main result is that the proposed algorithm almost surely converges to the desired mean $mu.$ We derive this convergence using a novel differential-inclusion-based two-timescale analysis.
  arXiv  Detail & Related papers  (2023-04-04T04:32:29Z)
• The Projected Covariance Measure for assumption-lean variable significance testing [3.8936058127056357]
  A simple but common approach is to specify a linear model, and then test whether the regression coefficient for $X$ is non-zero. We study the problem of testing the model-free null of conditional mean independence, i.e. that the conditional mean of $Y$ given $X$ and $Z$ does not depend on $X$. We propose a simple and general framework that can leverage flexible nonparametric or machine learning methods, such as additive models or random forests.
  arXiv  Detail & Related papers  (2022-11-03T17:55:50Z)
• On the Identifiability and Estimation of Causal Location-Scale Noise Models [122.65417012597754]
  We study the class of location-scale or heteroscedastic noise models (LSNMs) We show the causal direction is identifiable up to some pathological cases. We propose two estimators for LSNMs: an estimator based on (non-linear) feature maps, and one based on neural networks.
  arXiv  Detail & Related papers  (2022-10-13T17:18:59Z)
• Weighted-average quantile regression [1.0742675209112622]
  We introduce the weighted-average quantile regression framework, $int_Y|X(u)psi(u)du = X'beta$, where $Y$ is a dependent variable. We develop an estimator of the vector of parameters $beta$, where $T$ is the size of available sample.
  arXiv  Detail & Related papers  (2022-03-06T19:06:53Z)
• Mediated Uncoupled Learning: Learning Functions without Direct Input-output Correspondences [80.95776331769899]
  We consider the task of predicting $Y$ from $X$ when we have no paired data of them. A naive approach is to predict $U$ from $X$ using $S_X$ and then $Y$ from $U$ using $S_Y$. We propose a new method that avoids predicting $U$ but directly learns $Y = f(X)$ by training $f(X)$ with $S_X$ to predict $h(U)$.
  arXiv  Detail & Related papers  (2021-07-16T22:13:29Z)
• Simplest non-additive measures of quantum resources [77.34726150561087]
  We study measures that can be described by $cal E(rhootimes N) =E(e;N) ne Ne$.
  arXiv  Detail & Related papers  (2021-06-23T20:27:04Z)
• Learning and Sampling of Atomic Interventions from Observations [11.522442415989818]
  We study the problem of efficiently estimating the effect of an intervention on a single variable (atomic interventions) using observational samples in a causal Bayesian network. Our goal is to give algorithms that are efficient in both time and sample complexity in a non-parametric setting.
  arXiv  Detail & Related papers  (2020-02-11T07:15:32Z)
• Optimal rates for independence testing via $U$-statistic permutation tests [7.090165638014331]
  We study the problem of independence testing given independent and identically distributed pairs taking values in a $sigma$-finite, separable measure space. We first show that there is no valid test of independence that is uniformly consistent against alternatives of the form $f: D(f) geq rho2 $.
  arXiv  Detail & Related papers  (2020-01-15T19:04:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.