Clustering above Exponential Families with Tempered Exponential Measures

• URL: http://arxiv.org/abs/2211.02765v1
• Date: Fri, 4 Nov 2022 21:58:40 GMT
• Title: Clustering above Exponential Families with Tempered Exponential Measures
• Authors: Ehsan Amid, Richard Nock, Manfred Warmuth
• Abstract summary: Link with exponential families has allowed $k$-means clustering to be generalized to a wide variety of data generating distributions. Getting the framework to work above exponential families is important to lift roadblocks like the lack of robustness of some population minimizers carved in their axiomatization.
• Score: 28.532545355403123
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The link with exponential families has allowed $k$-means clustering to be generalized to a wide variety of data generating distributions in exponential families and clustering distortions among Bregman divergences. Getting the framework to work above exponential families is important to lift roadblocks like the lack of robustness of some population minimizers carved in their axiomatization. Current generalisations of exponential families like $q$-exponential families or even deformed exponential families fail at achieving the goal. In this paper, we provide a new attempt at getting the complete framework, grounded in a new generalisation of exponential families that we introduce, tempered exponential measures (TEM). TEMs keep the maximum entropy axiomatization framework of $q$-exponential families, but instead of normalizing the measure, normalize a dual called a co-distribution. Numerous interesting properties arise for clustering such as improved and controllable robustness for population minimizers, that keep a simple analytic form.

Related papers

• Almost Free: Self-concordance in Natural Exponential Families and an Application to Bandits [32.10916558109406]
  We show that optimistic algorithms for generalized linear bandits enjoy regret bounds that are both second-order and free of an exponential dependence on the bound of the problem parameter in the leading term. Ours is the first regret bound for generalized linear bandits with subexponential tails, broadening the class of problems to include Poisson, exponential and gamma bandits.
  arXiv  Detail & Related papers  (2024-10-01T22:42:19Z)
• Finite-Time Regret of Thompson Sampling Algorithms for Exponential Family Multi-Armed Bandits [88.21288104408556]
  We study the regret of Thompson sampling (TS) algorithms for exponential family bandits, where the reward distribution is from a one-dimensional exponential family. We propose a Thompson sampling, termed Expulli, which uses a novel sampling distribution to avoid the under-estimation of the optimal arm.
  arXiv  Detail & Related papers  (2022-06-07T18:08:21Z)
• Bregman Deviations of Generic Exponential Families [28.592975143984006]
  We revisit the method of mixture technique, also known as the Laplace method, to study the concentration phenomenon in generic exponential families. Our bound is time-uniform and makes appear a quantity extending the classical information gain to exponential families.
  arXiv  Detail & Related papers  (2022-01-18T20:50:05Z)
• Kernel Deformed Exponential Families for Sparse Continuous Attention [76.61129971916702]
  We show existence results for kernel exponential and deformed exponential families. Experiments show that kernel deformed exponential families can attend to multiple compact regions of the data domain.
  arXiv  Detail & Related papers  (2021-11-01T19:21:22Z)
• Exact Recovery in the General Hypergraph Stochastic Block Model [92.28929858529679]
  This paper investigates fundamental limits of exact recovery in the general d-uniform hypergraph block model (d-HSBM) We show that there exists a sharp threshold such that exact recovery is achievable above the threshold and impossible below it.
  arXiv  Detail & Related papers  (2021-05-11T03:39:08Z)
• An Online Learning Approach to Interpolation and Extrapolation in Domain Generalization [53.592597682854944]
  We recast generalization over sub-groups as an online game between a player minimizing risk and an adversary presenting new test. We show that ERM is provably minimax-optimal for both tasks.
  arXiv  Detail & Related papers  (2021-02-25T19:06:48Z)
• Likelihood Ratio Exponential Families [43.98796887171374]
  We use the geometric mixture path as an exponential family of distributions to analyze the thermodynamic variational objective (TVO) We extend these likelihood ratio exponential families to include solutions to rate-distortion (RD) optimization, the information bottleneck (IB) method, and recent rate-distortion-classification approaches.
  arXiv  Detail & Related papers  (2020-12-31T07:13:58Z)
• Flexible mean field variational inference using mixtures of non-overlapping exponential families [6.599344783327053]
  I show that using standard mean field variational inference can fail to produce sensible results for models with sparsity-inducing priors. I show that any mixture of a diffuse exponential family and a point mass at zero to model sparsity forms an exponential family.
  arXiv  Detail & Related papers  (2020-10-14T01:46:56Z)
• Relative Deviation Margin Bounds [55.22251993239944]
  We give two types of learning bounds, both distribution-dependent and valid for general families, in terms of the Rademacher complexity. We derive distribution-dependent generalization bounds for unbounded loss functions under the assumption of a finite moment.
  arXiv  Detail & Related papers  (2020-06-26T12:37:17Z)
• Semiparametric Nonlinear Bipartite Graph Representation Learning with Provable Guarantees [106.91654068632882]
  We consider the bipartite graph and formalize its representation learning problem as a statistical estimation problem of parameters in a semiparametric exponential family distribution. We show that the proposed objective is strongly convex in a neighborhood around the ground truth, so that a gradient descent-based method achieves linear convergence rate. Our estimator is robust to any model misspecification within the exponential family, which is validated in extensive experiments.
  arXiv  Detail & Related papers  (2020-03-02T16:40:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.