Bias-corrected estimator for intrinsic dimension and differential entropy--a visual multiscale approach

• URL: http://arxiv.org/abs/2004.14528v1
• Date: Thu, 30 Apr 2020 00:29:28 GMT
• Title: Bias-corrected estimator for intrinsic dimension and differential entropy--a visual multiscale approach
• Authors: Jugurta Montalv\~ao, J\^anio Canuto, Luiz Miranda
• Abstract summary: Intrinsic and differential entropy estimators are studied in this paper, including their systematic bias. A pragmatic approach for joint estimation and bias correction of these two fundamental measures is proposed. It is shown that both estimators can be complementary parts of a single approach, and that the simultaneous estimation of differential entropy and intrinsic dimension give meaning to each other.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Intrinsic dimension and differential entropy estimators are studied in this paper, including their systematic bias. A pragmatic approach for joint estimation and bias correction of these two fundamental measures is proposed. Shared steps on both estimators are highlighted, along with their useful consequences to data analysis. It is shown that both estimators can be complementary parts of a single approach, and that the simultaneous estimation of differential entropy and intrinsic dimension give meaning to each other, where estimates at different observation scales convey different perspectives of underlying manifolds. Experiments with synthetic and real datasets are presented to illustrate how to extract meaning from visual inspections, and how to compensate for biases.

Related papers

• Estimation of multiple mean vectors in high dimension [4.2466572124753]
  We endeavour to estimate numerous multi-dimensional means of various probability distributions on a common space based on independent samples. Our approach involves forming estimators through convex combinations of empirical means derived from these samples.
  arXiv  Detail & Related papers  (2024-03-22T08:42:41Z)
• Empirical fits to inclusive electron-carbon scattering data obtained by deep-learning methods [0.0]
  We obtain empirical fits to the electron-scattering cross sections for carbon over a broad kinematic region. We consider two different methods of obtaining such model-independent parametrizations and the corresponding uncertainties.
  arXiv  Detail & Related papers  (2023-12-28T17:03:17Z)
• Approximating Counterfactual Bounds while Fusing Observational, Biased and Randomised Data Sources [64.96984404868411]
  We address the problem of integrating data from multiple, possibly biased, observational and interventional studies. We show that the likelihood of the available data has no local maxima. We then show how the same approach can address the general case of multiple datasets.
  arXiv  Detail & Related papers  (2023-07-31T11:28:24Z)
• Causal Effect Estimation from Observational and Interventional Data Through Matrix Weighted Linear Estimators [11.384045395629123]
  We study causal effect estimation from a mixture of observational and interventional data. We show that the statistical efficiency in terms of expected squared error can be improved by combining estimators.
  arXiv  Detail & Related papers  (2023-06-09T16:16:53Z)
• Exploiting Observation Bias to Improve Matrix Completion [16.57405742112833]
  We consider a variant of matrix completion where entries are revealed in a biased manner. The goal is to exploit the shared information between the bias and the outcome of interest to improve predictions. We find that with this two-stage algorithm, the estimates have 30x smaller mean squared error compared to traditional matrix completion methods.
  arXiv  Detail & Related papers  (2023-06-07T20:48:35Z)
• Eigen Analysis of Self-Attention and its Reconstruction from Partial Computation [58.80806716024701]
  We study the global structure of attention scores computed using dot-product based self-attention. We find that most of the variation among attention scores lie in a low-dimensional eigenspace. We propose to compute scores only for a partial subset of token pairs, and use them to estimate scores for the remaining pairs.
  arXiv  Detail & Related papers  (2021-06-16T14:38:42Z)
• A similarity-based Bayesian mixture-of-experts model [0.5156484100374058]
  We present a new non-parametric mixture-of-experts model for multivariate regression problems. Using a conditionally specified model, predictions for out-of-sample inputs are based on similarities to each observed data point. Posterior inference is performed on the parameters of the mixture as well as the distance metric.
  arXiv  Detail & Related papers  (2020-12-03T18:08:30Z)
• Learning Disentangled Representations with Latent Variation Predictability [102.4163768995288]
  This paper defines the variation predictability of latent disentangled representations. Within an adversarial generation process, we encourage variation predictability by maximizing the mutual information between latent variations and corresponding image pairs. We develop an evaluation metric that does not rely on the ground-truth generative factors to measure the disentanglement of latent representations.
  arXiv  Detail & Related papers  (2020-07-25T08:54:26Z)
• On Disentangled Representations Learned From Correlated Data [59.41587388303554]
  We bridge the gap to real-world scenarios by analyzing the behavior of the most prominent disentanglement approaches on correlated data. We show that systematically induced correlations in the dataset are being learned and reflected in the latent representations. We also demonstrate how to resolve these latent correlations, either using weak supervision during training or by post-hoc correcting a pre-trained model with a small number of labels.
  arXiv  Detail & Related papers  (2020-06-14T12:47:34Z)
• Machine learning for causal inference: on the use of cross-fit estimators [77.34726150561087]
  Doubly-robust cross-fit estimators have been proposed to yield better statistical properties. We conducted a simulation study to assess the performance of several estimators for the average causal effect (ACE) When used with machine learning, the doubly-robust cross-fit estimators substantially outperformed all of the other estimators in terms of bias, variance, and confidence interval coverage.
  arXiv  Detail & Related papers  (2020-04-21T23:09:55Z)
• Estimating Treatment Effects with Observed Confounders and Mediators [25.338901482522648]
  Given a causal graph, the do-calculus can express treatment effects as functionals of the observational joint distribution that can be estimated empirically. Sometimes the do-calculus identifies multiple valid formulae, prompting us to compare the statistical properties of the corresponding estimators. In this paper, we investigate the over-identified scenario where both confounders and mediators are observed, rendering both estimators valid.
  arXiv  Detail & Related papers  (2020-03-26T15:50:25Z)
• Learning Overlapping Representations for the Estimation of Individualized Treatment Effects [97.42686600929211]
  Estimating the likely outcome of alternatives from observational data is a challenging problem. We show that algorithms that learn domain-invariant representations of inputs are often inappropriate. We develop a deep kernel regression algorithm and posterior regularization framework that substantially outperforms the state-of-the-art on a variety of benchmarks data sets.
  arXiv  Detail & Related papers  (2020-01-14T12:56:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.