Deep Archimedean Copulas
- URL: http://arxiv.org/abs/2012.03137v1
- Date: Sat, 5 Dec 2020 22:58:37 GMT
- Title: Deep Archimedean Copulas
- Authors: Chun Kai Ling, Fei Fang, J. Zico Kolter
- Abstract summary: ACNet is a novel differentiable neural network architecture that enforces structural properties.
We show that ACNet is able to both approximate common Archimedean Copulas and generate new copulas which may provide better fits to data.
- Score: 98.96141706464425
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: A central problem in machine learning and statistics is to model joint
densities of random variables from data. Copulas are joint cumulative
distribution functions with uniform marginal distributions and are used to
capture interdependencies in isolation from marginals. Copulas are widely used
within statistics, but have not gained traction in the context of modern deep
learning. In this paper, we introduce ACNet, a novel differentiable neural
network architecture that enforces structural properties and enables one to
learn an important class of copulas--Archimedean Copulas. Unlike Generative
Adversarial Networks, Variational Autoencoders, or Normalizing Flow methods,
which learn either densities or the generative process directly, ACNet learns a
generator of the copula, which implicitly defines the cumulative distribution
function of a joint distribution. We give a probabilistic interpretation of the
network parameters of ACNet and use this to derive a simple but efficient
sampling algorithm for the learned copula. Our experiments show that ACNet is
able to both approximate common Archimedean Copulas and generate new copulas
which may provide better fits to data.
Related papers
- Demolition and Reinforcement of Memories in Spin-Glass-like Neural
Networks [0.0]
The aim of this thesis is to understand the effectiveness of Unlearning in both associative memory models and generative models.
The selection of structured data enables an associative memory model to retrieve concepts as attractors of a neural dynamics with considerable basins of attraction.
A novel regularization technique for Boltzmann Machines is presented, proving to outperform previously developed methods in learning hidden probability distributions from data-sets.
arXiv Detail & Related papers (2024-03-04T23:12:42Z) - On Neural Architecture Inductive Biases for Relational Tasks [76.18938462270503]
We introduce a simple architecture based on similarity-distribution scores which we name Compositional Network generalization (CoRelNet)
We find that simple architectural choices can outperform existing models in out-of-distribution generalizations.
arXiv Detail & Related papers (2022-06-09T16:24:01Z) - Neural Copula: A unified framework for estimating generic
high-dimensional Copula functions [7.926596248777508]
The Copula is widely used to describe the relationship between the marginal distribution and joint distribution of random variables.
A novel neural network-based method (named Neural Copula) is proposed in this paper.
The effectiveness of the proposed method is evaluated on both real-world datasets and complex numerical simulations.
arXiv Detail & Related papers (2022-05-23T13:01:44Z) - Decomposing neural networks as mappings of correlation functions [57.52754806616669]
We study the mapping between probability distributions implemented by a deep feed-forward network.
We identify essential statistics in the data, as well as different information representations that can be used by neural networks.
arXiv Detail & Related papers (2022-02-10T09:30:31Z) - Learning with latent group sparsity via heat flow dynamics on networks [5.076419064097734]
Group or cluster structure on explanatory variables in machine learning problems is a very general phenomenon.
We contribute an approach to learning under such group structure, that does not require prior information on the group identities.
We demonstrate a procedure to construct such a network based on the available data.
arXiv Detail & Related papers (2022-01-20T17:45:57Z) - Geometry-Aware Hierarchical Bayesian Learning on Manifolds [5.182379239800725]
We propose a hierarchical Bayesian learning model for learning on manifold-valued vision data.
We first introduce a kernel with the properties of geometry-awareness and intra- Kernel convolution.
We then use a Gaussian process regression to organize the inputs and finally implement a hierarchical Bayesian network for the feature aggregation.
arXiv Detail & Related papers (2021-10-30T05:47:05Z) - Dynamic Inference with Neural Interpreters [72.90231306252007]
We present Neural Interpreters, an architecture that factorizes inference in a self-attention network as a system of modules.
inputs to the model are routed through a sequence of functions in a way that is end-to-end learned.
We show that Neural Interpreters perform on par with the vision transformer using fewer parameters, while being transferrable to a new task in a sample efficient manner.
arXiv Detail & Related papers (2021-10-12T23:22:45Z) - The Separation Capacity of Random Neural Networks [78.25060223808936]
We show that a sufficiently large two-layer ReLU-network with standard Gaussian weights and uniformly distributed biases can solve this problem with high probability.
We quantify the relevant structure of the data in terms of a novel notion of mutual complexity.
arXiv Detail & Related papers (2021-07-31T10:25:26Z) - Network Classifiers Based on Social Learning [71.86764107527812]
We propose a new way of combining independently trained classifiers over space and time.
The proposed architecture is able to improve prediction performance over time with unlabeled data.
We show that this strategy results in consistent learning with high probability, and it yields a robust structure against poorly trained classifiers.
arXiv Detail & Related papers (2020-10-23T11:18:20Z) - Hierarchical Gaussian Process Priors for Bayesian Neural Network Weights [16.538973310830414]
A desirable class of priors would represent weights compactly, capture correlations between weights, and allow inclusion of prior knowledge.
This paper introduces two innovations: (i) a process-based hierarchical model for network weights based on unit embeddings that can flexibly encode correlated weight structures, and (ii) input-dependent versions of these weight priors that can provide convenient ways to regularize the function space.
We show these models provide desirable test-time uncertainty estimates on out-of-distribution data, demonstrate cases of modeling inductive biases for neural networks with kernels, and demonstrate competitive predictive performance on an active learning benchmark
arXiv Detail & Related papers (2020-02-10T07:19:52Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.