Marginal Tail-Adaptive Normalizing Flows

• URL: http://arxiv.org/abs/2206.10311v1
• Date: Tue, 21 Jun 2022 12:34:36 GMT
• Title: Marginal Tail-Adaptive Normalizing Flows
• Authors: Mike Laszkiewicz, Johannes Lederer, Asja Fischer
• Abstract summary: This paper focuses on improving the ability of normalizing flows to correctly capture the tail behavior. We prove that the marginal tailedness of an autoregressive flow can be controlled via the tailedness of the marginals of its base distribution. An empirical analysis shows that the proposed method improves on the accuracy -- especially on the tails of the distribution -- and is able to generate heavy-tailed data.
• Score: 15.732950126814089
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Learning the tail behavior of a distribution is a notoriously difficult problem. By definition, the number of samples from the tail is small, and deep generative models, such as normalizing flows, tend to concentrate on learning the body of the distribution. In this paper, we focus on improving the ability of normalizing flows to correctly capture the tail behavior and, thus, form more accurate models. We prove that the marginal tailedness of an autoregressive flow can be controlled via the tailedness of the marginals of its base distribution. This theoretical insight leads us to a novel type of flows based on flexible base distributions and data-driven linear layers. An empirical analysis shows that the proposed method improves on the accuracy -- especially on the tails of the distribution -- and is able to generate heavy-tailed data. We demonstrate its application on a weather and climate example, in which capturing the tail behavior is essential.

Related papers

• Flexible Tails for Normalizing Flows [0.658372523529902]
  A popular current solution to this problem is to use a heavy tailed base distribution. We argue this can lead to poor performance due to the difficulty of optimising neural networks, such as normalizing flows, under heavy tailed input. We propose an alternative: use a Gaussian base distribution and a final transformation layer which can produce heavy tails.
  arXiv  Detail & Related papers  (2024-06-22T13:44:01Z)
• Rejection via Learning Density Ratios [50.91522897152437]
  Classification with rejection emerges as a learning paradigm which allows models to abstain from making predictions. We propose a different distributional perspective, where we seek to find an idealized data distribution which maximizes a pretrained model's performance. Our framework is tested empirically over clean and noisy datasets.
  arXiv  Detail & Related papers  (2024-05-29T01:32:17Z)
• Out-of-distribution detection using normalizing flows on the data manifold [3.725042082196983]
  This study investigates the effect of manifold learning using normalizing flows on out-of-distribution detection. We show that manifold learning improves the out-of-distribution detection ability of a class of likelihood-based models known as normalizing flows.
  arXiv  Detail & Related papers  (2023-08-26T07:35:16Z)
• A Heavy-Tailed Algebra for Probabilistic Programming [53.32246823168763]
  We propose a systematic approach for analyzing the tails of random variables. We show how this approach can be used during the static analysis (before drawing samples) pass of a probabilistic programming language compiler. Our empirical results confirm that inference algorithms that leverage our heavy-tailed algebra attain superior performance across a number of density modeling and variational inference tasks.
  arXiv  Detail & Related papers  (2023-06-15T16:37:36Z)
• Tensorizing flows: a tool for variational inference [0.0]
  We introduce an extension of normalizing flows in which the Gaussian reference is replaced with a reference distribution constructed via a tensor network. We show that by combining flows with tensor networks on difficult variational inference tasks, we can improve on the results obtained by using either tool without the other.
  arXiv  Detail & Related papers  (2023-05-03T23:42:22Z)
• ManiFlow: Implicitly Representing Manifolds with Normalizing Flows [145.9820993054072]
  Normalizing Flows (NFs) are flexible explicit generative models that have been shown to accurately model complex real-world data distributions. We propose an optimization objective that recovers the most likely point on the manifold given a sample from the perturbed distribution. Finally, we focus on 3D point clouds for which we utilize the explicit nature of NFs, i.e. surface normals extracted from the gradient of the log-likelihood and the log-likelihood itself.
  arXiv  Detail & Related papers  (2022-08-18T16:07:59Z)
• Fat-Tailed Variational Inference with Anisotropic Tail Adaptive Flows [53.32246823168763]
  Fat-tailed densities commonly arise as posterior and marginal distributions in robust models and scale mixtures. We first improve previous theory on tails of Lipschitz flows by quantifying how tails affect the rate of tail decay. We then develop an alternative theory for tail parameters which is sensitive to tail-anisotropy.
  arXiv  Detail & Related papers  (2022-05-16T18:03:41Z)
• Nonlinear Isometric Manifold Learning for Injective Normalizing Flows [58.720142291102135]
  We use isometries to separate manifold learning and density estimation. We also employ autoencoders to design embeddings with explicit inverses that do not distort the probability distribution.
  arXiv  Detail & Related papers  (2022-03-08T08:57:43Z)
• Copula-Based Normalizing Flows [8.894004971395546]
  We generalize the base distribution to a more elaborate copula distribution to capture the properties of the target distribution more accurately. Our results suggest that the improvements are related to an increased local Lipschitz-stability of the learned flow.
  arXiv  Detail & Related papers  (2021-07-15T14:22:28Z)
• Semi-Supervised Learning with Normalizing Flows [54.376602201489995]
  FlowGMM is an end-to-end approach to generative semi supervised learning with normalizing flows. We show promising results on a wide range of applications, including AG-News and Yahoo Answers text data.
  arXiv  Detail & Related papers  (2019-12-30T17:36:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.