A Generalization of Spatial Monte Carlo Integration

• URL: http://arxiv.org/abs/2009.02165v2
• Date: Thu, 17 Sep 2020 01:02:56 GMT
• Title: A Generalization of Spatial Monte Carlo Integration
• Authors: Muneki Yasuda and Kei Uchizawa
• Abstract summary: Spatial Monte Carlo integration (SMCI) is an extension of standard Monte Carlo integration and can approximate expectations on Markov random fields with high accuracy. A new Boltzmann machine learning method based on SMCI is proposed, which is obtained by combining SMCI and the persistent contrastive divergence.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Spatial Monte Carlo integration (SMCI) is an extension of standard Monte Carlo integration and can approximate expectations on Markov random fields with high accuracy. SMCI was applied to pairwise Boltzmann machine (PBM) learning, with superior results to those from some existing methods. The approximation level of SMCI can be changed, and it was proved that a higher-order approximation of SMCI is statistically more accurate than a lower-order approximation. However, SMCI as proposed in the previous studies suffers from a limitation that prevents the application of a higher-order method to dense systems. This study makes two different contributions as follows. A generalization of SMCI (called generalized SMCI (GSMCI)) is proposed, which allows relaxation of the above-mentioned limitation; moreover, a statistical accuracy bound of GSMCI is proved. This is the first contribution of this study. A new PBM learning method based on SMCI is proposed, which is obtained by combining SMCI and the persistent contrastive divergence. The proposed learning method greatly improves the accuracy of learning. This is the second contribution of this study.

Related papers

• Computing Low-Entropy Couplings for Large-Support Distributions [53.00113867130712]
  Minimum-entropy coupling has applications in areas such as causality and steganography. Existing algorithms are either computationally intractable for large-support distributions or limited to specific distribution types. This work addresses these limitations by unifying a prior family of iterative MEC approaches into a generalized partition-based formalism.
  arXiv  Detail & Related papers  (2024-05-29T21:54:51Z)
• Unleashing Network Potentials for Semantic Scene Completion [50.95486458217653]
  This paper proposes a novel SSC framework - Adrial Modality Modulation Network (AMMNet) AMMNet introduces two core modules: a cross-modal modulation enabling the interdependence of gradient flows between modalities, and a customized adversarial training scheme leveraging dynamic gradient competition. Extensive experimental results demonstrate that AMMNet outperforms state-of-the-art SSC methods by a large margin.
  arXiv  Detail & Related papers  (2024-03-12T11:48:49Z)
• Combining Normalizing Flows and Quasi-Monte Carlo [0.0]
  Recent advances in machine learning have led to the development of new methods for enhancing Monte Carlo methods. We demonstrate through numerical experiments that this combination can lead to an estimator with significantly lower variance than if the flow was sampled with a classic Monte Carlo.
  arXiv  Detail & Related papers  (2024-01-11T14:17:06Z)
• Federated Multi-Sequence Stochastic Approximation with Local Hypergradient Estimation [28.83712379658548]
  We develop FedMSA, the first federated approximation algorithm for multiple coupled sequences (MSA) FedMSA enables the provable estimation of hypergradients in BLO and MCO via local client updates. We provide experiments that support our theory and demonstrate the empirical benefits of FedMSA.
  arXiv  Detail & Related papers  (2023-06-02T16:17:43Z)
• Composite Spatial Monte Carlo Integration Based on Generalized Least Squares [0.0]
  Spatial Monte Carlo integration (SMCI) is a sampling-based approximation. A new effective method is proposed by combining multiple SMCI estimators. The results indicate that the proposed method can be effective in the inverse Ising problem (or Boltzmann machine learning)
  arXiv  Detail & Related papers  (2022-04-07T06:35:13Z)
• Counterfactual Maximum Likelihood Estimation for Training Deep Networks [83.44219640437657]
  Deep learning models are prone to learning spurious correlations that should not be learned as predictive clues. We propose a causality-based training framework to reduce the spurious correlations caused by observable confounders. We conduct experiments on two real-world tasks: Natural Language Inference (NLI) and Image Captioning.
  arXiv  Detail & Related papers  (2021-06-07T17:47:16Z)
• Sampling in Combinatorial Spaces with SurVAE Flow Augmented MCMC [83.48593305367523]
  Hybrid Monte Carlo is a powerful Markov Chain Monte Carlo method for sampling from complex continuous distributions. We introduce a new approach based on augmenting Monte Carlo methods with SurVAE Flows to sample from discrete distributions. We demonstrate the efficacy of our algorithm on a range of examples from statistics, computational physics and machine learning, and observe improvements compared to alternative algorithms.
  arXiv  Detail & Related papers  (2021-02-04T02:21:08Z)
• Spatial Monte Carlo Integration with Annealed Importance Sampling [0.45687771576879593]
  A new method is proposed to evaluate the expectations on Ising models combining AIS and SMCI. The proposed method performs efficiently in both high- and low-temperature regions.
  arXiv  Detail & Related papers  (2020-12-21T09:26:40Z)
• Semi-Supervised Learning with Variational Bayesian Inference and Maximum Uncertainty Regularization [62.21716612888669]
  We propose two generic methods for improving semi-supervised learning (SSL) The first integrates weight perturbation (WP) into existing "consistency regularization" (CR) based methods. The second method proposes a novel consistency loss called "maximum uncertainty regularization" (MUR)
  arXiv  Detail & Related papers  (2020-12-03T09:49:35Z)
• Joint Stochastic Approximation and Its Application to Learning Discrete Latent Variable Models [19.07718284287928]
  We show that the difficulty of obtaining reliable gradients for the inference model and the drawback of indirectly optimizing the target log-likelihood can be gracefully addressed. We propose to directly maximize the target log-likelihood and simultaneously minimize the inclusive divergence between the posterior and the inference model. The resulting learning algorithm is called joint SA (JSA)
  arXiv  Detail & Related papers  (2020-05-28T13:50:08Z)
• Theoretical Convergence of Multi-Step Model-Agnostic Meta-Learning [63.64636047748605]
  We develop a new theoretical framework to provide convergence guarantee for the general multi-step MAML algorithm. In particular, our results suggest that an inner-stage step needs to be chosen inversely proportional to $N$ of inner-stage steps in order for $N$ MAML to have guaranteed convergence.
  arXiv  Detail & Related papers  (2020-02-18T19:17:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.