Measuring Statistical Dependencies via Maximum Norm and Characteristic Functions

• URL: http://arxiv.org/abs/2208.07934v1
• Date: Tue, 16 Aug 2022 20:24:31 GMT
• Title: Measuring Statistical Dependencies via Maximum Norm and Characteristic Functions
• Authors: Povilas Daniu\v{s}is, Shubham Juneja, Lukas Kuzma, Virginijus Marcinkevi\v{c}ius
• Abstract summary: We propose a statistical dependence measure based on the maximum-norm of the difference between joint and product-marginal characteristic functions. The proposed measure can detect arbitrary statistical dependence between two random vectors of possibly different dimensions. We conduct experiments both with simulated and real data.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper, we focus on the problem of statistical dependence estimation using characteristic functions. We propose a statistical dependence measure, based on the maximum-norm of the difference between joint and product-marginal characteristic functions. The proposed measure can detect arbitrary statistical dependence between two random vectors of possibly different dimensions, is differentiable, and easily integrable into modern machine learning and deep learning pipelines. We also conduct experiments both with simulated and real data. Our simulations show, that the proposed method can measure statistical dependencies in high-dimensional, non-linear data, and is less affected by the curse of dimensionality, compared to the previous work in this line of research. The experiments with real data demonstrate the potential applicability of our statistical measure for two different empirical inference scenarios, showing statistically significant improvement in the performance characteristics when applied for supervised feature extraction and deep neural network regularization. In addition, we provide a link to the accompanying open-source repository https://bit.ly/3d4ch5I.

Related papers

• Statistical Inference for Temporal Difference Learning with Linear Function Approximation [62.69448336714418]
  Temporal Difference (TD) learning, arguably the most widely used for policy evaluation, serves as a natural framework for this purpose. In this paper, we study the consistency properties of TD learning with Polyak-Ruppert averaging and linear function approximation, and obtain three significant improvements over existing results.
  arXiv  Detail & Related papers  (2024-10-21T15:34:44Z)
• Statistical Efficiency of Score Matching: The View from Isoperimetry [96.65637602827942]
  We show a tight connection between statistical efficiency of score matching and the isoperimetric properties of the distribution being estimated. We formalize these results both in the sample regime and in the finite regime.
  arXiv  Detail & Related papers  (2022-10-03T06:09:01Z)
• Training Normalizing Flows from Dependent Data [31.42053454078623]
  We propose a likelihood objective of normalizing flows incorporating dependencies between the data points. We show that respecting dependencies between observations can improve empirical results on both synthetic and real-world data.
  arXiv  Detail & Related papers  (2022-09-29T16:50:34Z)
• Data-Driven Influence Functions for Optimization-Based Causal Inference [105.5385525290466]
  We study a constructive algorithm that approximates Gateaux derivatives for statistical functionals by finite differencing. We study the case where probability distributions are not known a priori but need to be estimated from data.
  arXiv  Detail & Related papers  (2022-08-29T16:16:22Z)
• Statistical inference of travelers' route choice preferences with system-level data [4.120057972557892]
  We develop a methodology to estimate travelers' utility functions with multiple attributes using system-level data. Experiments on synthetic data show that the coefficients are consistently recovered and that hypothesis tests are a reliable statistic to identify which attributes are determinants of travelers' route choices. The methodology is also deployed at a large scale using real Fresnoworld multisource data collected during the COVID outbreak.
  arXiv  Detail & Related papers  (2022-04-23T00:38:32Z)
• Uncertainty Modeling for Out-of-Distribution Generalization [56.957731893992495]
  We argue that the feature statistics can be properly manipulated to improve the generalization ability of deep learning models. Common methods often consider the feature statistics as deterministic values measured from the learned features. We improve the network generalization ability by modeling the uncertainty of domain shifts with synthesized feature statistics during training.
  arXiv  Detail & Related papers  (2022-02-08T16:09:12Z)
• Differential privacy and robust statistics in high dimensions [49.50869296871643]
  High-dimensional Propose-Test-Release (HPTR) builds upon three crucial components: the exponential mechanism, robust statistics, and the Propose-Test-Release mechanism. We show that HPTR nearly achieves the optimal sample complexity under several scenarios studied in the literature.
  arXiv  Detail & Related papers  (2021-11-12T06:36:40Z)
• Efficient Multidimensional Functional Data Analysis Using Marginal Product Basis Systems [2.4554686192257424]
  We propose a framework for learning continuous representations from a sample of multidimensional functional data. We show that the resulting estimation problem can be solved efficiently by the tensor decomposition. We conclude with a real data application in neuroimaging.
  arXiv  Detail & Related papers  (2021-07-30T16:02:15Z)
• Measuring Dependence with Matrix-based Entropy Functional [21.713076360132195]
  Measuring the dependence of data plays a central role in statistics and machine learning. We propose two measures, namely the matrix-based normalized total correlation ($T_alpha*$) and the matrix-based normalized dual total correlation ($D_alpha*$) We show that our measures are differentiable and statistically more powerful than prevalent ones.
  arXiv  Detail & Related papers  (2021-01-25T15:18:16Z)
• Neural Methods for Point-wise Dependency Estimation [129.93860669802046]
  We focus on estimating point-wise dependency (PD), which quantitatively measures how likely two outcomes co-occur. We demonstrate the effectiveness of our approaches in 1) MI estimation, 2) self-supervised representation learning, and 3) cross-modal retrieval task.
  arXiv  Detail & Related papers  (2020-06-09T23:26:15Z)
• Measuring the Discrepancy between Conditional Distributions: Methods, Properties and Applications [18.293397644865458]
  We propose a simple yet powerful test statistic to quantify the discrepancy between two conditional distributions. The new statistic avoids the explicit estimation of the underlying distributions in highdimensional space. It inherits the merits of the correntropy function to explicitly incorporate high-order statistics in the data.
  arXiv  Detail & Related papers  (2020-05-05T14:03:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.