Learning conditional distributions on continuous spaces

• URL: http://arxiv.org/abs/2406.09375v1
• Date: Thu, 13 Jun 2024 17:53:47 GMT
• Title: Learning conditional distributions on continuous spaces
• Authors: Cyril Bénézet, Ziteng Cheng, Sebastian Jaimungal,
• Abstract summary: We investigate sample-based learning of conditional distributions on multi-dimensional unit boxes. We employ two distinct clustering schemes: one based on a fixed-radius ball and the other on nearest neighbors. We propose to incorporate the nearest neighbors method into neural network training, as our empirical analysis indicates it has better performance in practice.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We investigate sample-based learning of conditional distributions on multi-dimensional unit boxes, allowing for different dimensions of the feature and target spaces. Our approach involves clustering data near varying query points in the feature space to create empirical measures in the target space. We employ two distinct clustering schemes: one based on a fixed-radius ball and the other on nearest neighbors. We establish upper bounds for the convergence rates of both methods and, from these bounds, deduce optimal configurations for the radius and the number of neighbors. We propose to incorporate the nearest neighbors method into neural network training, as our empirical analysis indicates it has better performance in practice. For efficiency, our training process utilizes approximate nearest neighbors search with random binary space partitioning. Additionally, we employ the Sinkhorn algorithm and a sparsity-enforced transport plan. Our empirical findings demonstrate that, with a suitably designed structure, the neural network has the ability to adapt to a suitable level of Lipschitz continuity locally. For reproducibility, our code is available at \url{https://github.com/zcheng-a/LCD_kNN}.

Related papers

• Adaptive $k$-nearest neighbor classifier based on the local estimation of the shape operator [49.87315310656657]
  We introduce a new adaptive $k$-nearest neighbours ($kK$-NN) algorithm that explores the local curvature at a sample to adaptively defining the neighborhood size. Results on many real-world datasets indicate that the new $kK$-NN algorithm yields superior balanced accuracy compared to the established $k$-NN method.
  arXiv  Detail & Related papers  (2024-09-08T13:08:45Z)
• Point Cloud Classification via Deep Set Linearized Optimal Transport [51.99765487172328]
  We introduce Deep Set Linearized Optimal Transport, an algorithm designed for the efficient simultaneous embedding of point clouds into an $L2-$space. This embedding preserves specific low-dimensional structures within the Wasserstein space while constructing a classifier to distinguish between various classes of point clouds. We showcase the advantages of our algorithm over the standard deep set approach through experiments on a flow dataset with a limited number of labeled point clouds.
  arXiv  Detail & Related papers  (2024-01-02T23:26:33Z)
• Combating Mode Collapse in GANs via Manifold Entropy Estimation [70.06639443446545]
  Generative Adversarial Networks (GANs) have shown compelling results in various tasks and applications. We propose a novel training pipeline to address the mode collapse issue of GANs.
  arXiv  Detail & Related papers  (2022-08-25T12:33:31Z)
• Decentralized Gossip-Based Stochastic Bilevel Optimization over Communication Networks [42.76623191830371]
  We propose a gossip-based distributed bilevel optimization algorithm. Agents can solve both networked and outer problems in a single time. Our algorithm achieves the state-of-the-art efficiency and test accuracy.
  arXiv  Detail & Related papers  (2022-06-22T06:38:54Z)
• STEM: A Stochastic Two-Sided Momentum Algorithm Achieving Near-Optimal Sample and Communication Complexities for Federated Learning [58.6792963686231]
  Federated Learning (FL) refers to the paradigm where multiple worker nodes (WNs) build a joint model by using local data. It is not clear how to choose the WNs' minimum update directions, the first minibatch sizes, and the local update frequency. We show that there is a trade-off curve between local update frequencies and local mini sizes, on which the above complexities can be maintained.
  arXiv  Detail & Related papers  (2021-06-19T06:13:45Z)
• Hyperdimensional Computing for Efficient Distributed Classification with Randomized Neural Networks [5.942847925681103]
  We study distributed classification, which can be employed in situations were data cannot be stored at a central location nor shared. We propose a more efficient solution for distributed classification by making use of a lossy compression approach applied when sharing the local classifiers with other agents.
  arXiv  Detail & Related papers  (2021-06-02T01:33:56Z)
• Partition-Guided GANs [63.980473635585234]
  We design a partitioner that breaks the space into smaller regions, each having a simpler distribution, and training a different generator for each partition. This is done in an unsupervised manner without requiring any labels. Experimental results on various standard benchmarks show that the proposed unsupervised model outperforms several recent methods.
  arXiv  Detail & Related papers  (2021-04-02T00:06:53Z)
• Learning Neural Network Subspaces [74.44457651546728]
  Recent observations have advanced our understanding of the neural network optimization landscape. With a similar computational cost as training one model, we learn lines, curves, and simplexes of high-accuracy neural networks. With a similar computational cost as training one model, we learn lines, curves, and simplexes of high-accuracy neural networks.
  arXiv  Detail & Related papers  (2021-02-20T23:26:58Z)
• Communication-Efficient Sampling for Distributed Training of Graph Convolutional Networks [3.075766050800645]
  Training Graph Convolutional Networks (GCNs) is expensive as it needs to aggregate data from neighboring nodes. Previous works have proposed various neighbor sampling methods that estimate the aggregation result based on a small number of sampled neighbors. We present an algorithm that determines the local sampling probabilities and makes sure our skewed neighbor sampling does not affect much the convergence of the training.
  arXiv  Detail & Related papers  (2021-01-19T16:12:44Z)
• Adaptive quadrature schemes for Bayesian inference via active learning [0.0]
  We propose novel adaptive quadrature schemes based on an active learning procedure. We consider an interpolative approach for building a surrogate density, combining it with Monte Carlo sampling methods and other quadrature rules. Numerical results show the advantage of the proposed approach, including a challenging inference problem in an astronomic model.
  arXiv  Detail & Related papers  (2020-05-31T15:02:32Z)
• One Size Fits All: Can We Train One Denoiser for All Noise Levels? [13.46272057205994]
  It is often preferred to train one neural network estimator and apply it to all noise levels. The de facto protocol is to train the estimator with noisy samples whose noise are uniformly distributed. This paper addresses the sample problem from a minimax risk optimization perspective.
  arXiv  Detail & Related papers  (2020-05-19T17:56:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.