Improving Local Identifiability in Probabilistic Box Embeddings

• URL: http://arxiv.org/abs/2010.04831v2
• Date: Thu, 29 Oct 2020 01:39:49 GMT
• Title: Improving Local Identifiability in Probabilistic Box Embeddings
• Authors: Shib Sankar Dasgupta, Michael Boratko, Dongxu Zhang, Luke Vilnis, Xiang Lorraine Li, Andrew McCallum
• Abstract summary: Box embeddings are a promising example of such an embedding, where objects are represented by n-dimensional hyperrectangles. The benefits of geometric embeddings also introduce a problem of local identifiability, where whole neighborhoods of parameters result in equivalent loss. We show that the calculation of the expected intersection involves all parameters, and we demonstrate experimentally that this drastically improves the ability of such models to learn.
• Score: 39.354564663134234
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Geometric embeddings have recently received attention for their natural ability to represent transitive asymmetric relations via containment. Box embeddings, where objects are represented by n-dimensional hyperrectangles, are a particularly promising example of such an embedding as they are closed under intersection and their volume can be calculated easily, allowing them to naturally represent calibrated probability distributions. The benefits of geometric embeddings also introduce a problem of local identifiability, however, where whole neighborhoods of parameters result in equivalent loss which impedes learning. Prior work addressed some of these issues by using an approximation to Gaussian convolution over the box parameters, however, this intersection operation also increases the sparsity of the gradient. In this work, we model the box parameters with min and max Gumbel distributions, which were chosen such that space is still closed under the operation of the intersection. The calculation of the expected intersection volume involves all parameters, and we demonstrate experimentally that this drastically improves the ability of such models to learn.

Related papers

• Optimal function estimation with photonic quantum sensor networks [0.0]
  We solve the problem of optimally measuring an analytic function of unknown local parameters each linearly coupled to a qubit sensor. In particular, we derive lower bounds on the achievable mean square error in estimating a linear function of either local phase shifts or quadrature displacements. For quadrature displacements, we extend bounds to the case of arbitrary linear functions.
  arXiv  Detail & Related papers  (2024-01-29T19:00:00Z)
• Intrinsic Bayesian Cramér-Rao Bound with an Application to Covariance Matrix Estimation [49.67011673289242]
  This paper presents a new performance bound for estimation problems where the parameter to estimate lies in a smooth manifold. It induces a geometry for the parameter manifold, as well as an intrinsic notion of the estimation error measure.
  arXiv  Detail & Related papers  (2023-11-08T15:17:13Z)
• Parameterizing density operators with arbitrary symmetries to gain advantage in quantum state estimation [0.0]
  We show how to parameterize a density matrix that has an arbitrary symmetry, knowing the generators of the Lie algebra. In addition, we run numerical experiments and apply these parameterizations to quantum state estimation of states with different symmetries.
  arXiv  Detail & Related papers  (2022-08-13T01:24:03Z)
• Characterization of variational quantum algorithms using free fermions [0.0]
  We numerically study the interplay between these symmetries and the locality of the target state. We find that the number of iterations to converge to the solution scales linearly with system size.
  arXiv  Detail & Related papers  (2022-06-13T18:11:16Z)
• Optimal oracle inequalities for solving projected fixed-point equations [53.31620399640334]
  We study methods that use a collection of random observations to compute approximate solutions by searching over a known low-dimensional subspace of the Hilbert space. We show how our results precisely characterize the error of a class of temporal difference learning methods for the policy evaluation problem with linear function approximation.
  arXiv  Detail & Related papers  (2020-12-09T20:19:32Z)
• A numerical framework for elastic surface matching, comparison, and interpolation [10.09712608508383]
  Surface comparison and matching is a challenging problem in computer vision. In this paper, we take an alternative approach which bypasses the direct estimation of reparametrizations. By avoiding altogether the need for reparametrizations, it provides the flexibility to deal with simplicial meshes.
  arXiv  Detail & Related papers  (2020-06-20T20:35:59Z)
• Variational Inference as Iterative Projection in a Bayesian Hilbert Space with Application to Robotic State Estimation [14.670851095242451]
  Variational Bayesian inference is an important machine-learning tool that finds application from statistics to robotics. We show that variational inference based on KL divergence can amount to iterative projection, in the Euclidean sense, of the Bayesian posterior onto a subspace corresponding to the selected approximation family.
  arXiv  Detail & Related papers  (2020-05-14T21:33:31Z)
• Asymptotic Analysis of an Ensemble of Randomly Projected Linear Discriminants [94.46276668068327]
  In [1], an ensemble of randomly projected linear discriminants is used to classify datasets. We develop a consistent estimator of the misclassification probability as an alternative to the computationally-costly cross-validation estimator. We also demonstrate the use of our estimator for tuning the projection dimension on both real and synthetic data.
  arXiv  Detail & Related papers  (2020-04-17T12:47:04Z)
• Spatially Adaptive Inference with Stochastic Feature Sampling and Interpolation [72.40827239394565]
  We propose to compute features only at sparsely sampled locations. We then densely reconstruct the feature map with an efficient procedure. The presented network is experimentally shown to save substantial computation while maintaining accuracy over a variety of computer vision tasks.
  arXiv  Detail & Related papers  (2020-03-19T15:36:31Z)
• Convex Geometry and Duality of Over-parameterized Neural Networks [70.15611146583068]
  We develop a convex analytic approach to analyze finite width two-layer ReLU networks. We show that an optimal solution to the regularized training problem can be characterized as extreme points of a convex set. In higher dimensions, we show that the training problem can be cast as a finite dimensional convex problem with infinitely many constraints.
  arXiv  Detail & Related papers  (2020-02-25T23:05:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.