Joint-stochastic-approximation Random Fields with Application to Semi-supervised Learning

• URL: http://arxiv.org/abs/2505.20330v1
• Date: Sat, 24 May 2025 07:04:32 GMT
• Title: Joint-stochastic-approximation Random Fields with Application to Semi-supervised Learning
• Authors: Yunfu Song, Zhijian Ou,
• Abstract summary: We present joint-stochastic-approximation random fields (JRFs) -- a new family of algorithms for building deep undirected generative models.<n>It is found through synthetic experiments that JRFs work well in balancing mode covering and mode missing, and match the empirical data distribution well.
• Score: 10.293017518216908
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Our examination of deep generative models (DGMs) developed for semi-supervised learning (SSL), mainly GANs and VAEs, reveals two problems. First, mode missing and mode covering phenomenons are observed in genertion with GANs and VAEs. Second, there exists an awkward conflict between good classification and good generation in SSL by employing directed generative models. To address these problems, we formally present joint-stochastic-approximation random fields (JRFs) -- a new family of algorithms for building deep undirected generative models, with application to SSL. It is found through synthetic experiments that JRFs work well in balancing mode covering and mode missing, and match the empirical data distribution well. Empirically, JRFs achieve good classification results comparable to the state-of-art methods on widely adopted datasets -- MNIST, SVHN, and CIFAR-10 in SSL, and simultaneously perform good generation.

Related papers

• CLIP Meets Diffusion: A Synergistic Approach to Anomaly Detection [54.85000884785013]
  Anomaly detection is a complex problem due to the ambiguity in defining anomalies, the diversity of anomaly types, and the scarcity of training data.<n>We propose CLIPfusion, a method that leverages both discriminative and generative foundation models.<n>We believe that our method underscores the effectiveness of multi-modal and multi-model fusion in tackling the multifaceted challenges of anomaly detection.
  arXiv  Detail & Related papers  (2025-06-13T13:30:15Z)
• Joint-stochastic-approximation Autoencoders with Application to Semi-supervised Learning [16.625057220045292]
  We present Joint-stochastic-approximation (JSA) autoencoders - a new family of algorithms for building deep directed generative models.<n> JSA learning algorithm directly maximizes the data log-likelihood and simultaneously minimizes the inclusive KL divergence between the posteriori and the inference model.<n>We empirically show that JSA autoencoders with discrete latent space achieve comparable performance to other state-of-the-art DGMs with continuous latent space in semi-supervised tasks.
  arXiv  Detail & Related papers  (2025-05-24T06:52:23Z)
• High-Performance Few-Shot Segmentation with Foundation Models: An Empirical Study [64.06777376676513]
  We develop a few-shot segmentation (FSS) framework based on foundation models. To be specific, we propose a simple approach to extract implicit knowledge from foundation models to construct coarse correspondence. Experiments on two widely used datasets demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-09-10T08:04:11Z)
• A Hybrid of Generative and Discriminative Models Based on the Gaussian-coupled Softmax Layer [5.33024001730262]
  We propose a method to train a hybrid of discriminative and generative models in a single neural network. We demonstrate that the proposed hybrid model can be applied to semi-supervised learning and confidence calibration.
  arXiv  Detail & Related papers  (2023-05-10T05:48:22Z)
• Language as a Latent Sequence: deep latent variable models for semi-supervised paraphrase generation [47.33223015862104]
  We present a novel unsupervised model named variational sequence auto-encoding reconstruction (VSAR), which performs latent sequence inference given an observed text. To leverage information from text pairs, we additionally introduce a novel supervised model we call dual directional learning (DDL), which is designed to integrate with our proposed VSAR model. Our empirical evaluations suggest that the combined model yields competitive performance against the state-of-the-art supervised baselines on complete data.
  arXiv  Detail & Related papers  (2023-01-05T19:35:30Z)
• Disentangled Generation with Information Bottleneck for Few-Shot Learning [21.131911207010376]
  Few-shot learning, which aims to classify unseen classes with few samples, is challenging due to data scarcity. We propose a novel Information Bottleneck (IB) based Disentangled Generation Framework (DisGenIB) DisGenIB can simultaneously guarantee the discrimination and diversity of generated samples.
  arXiv  Detail & Related papers  (2022-11-29T13:29:36Z)
• Riemannian Score-Based Generative Modeling [56.20669989459281]
  We introduce score-based generative models (SGMs) demonstrating remarkable empirical performance. Current SGMs make the underlying assumption that the data is supported on a Euclidean manifold with flat geometry. This prevents the use of these models for applications in robotics, geoscience or protein modeling.
  arXiv  Detail & Related papers  (2022-02-06T11:57:39Z)
• Score-based Generative Modeling in Latent Space [93.8985523558869]
  Score-based generative models (SGMs) have recently demonstrated impressive results in terms of both sample quality and distribution coverage. Here, we propose the Latent Score-based Generative Model (LSGM), a novel approach that trains SGMs in a latent space. Moving from data to latent space allows us to train more expressive generative models, apply SGMs to non-continuous data, and learn smoother SGMs in a smaller space.
  arXiv  Detail & Related papers  (2021-06-10T17:26:35Z)
• Continual Learning with Fully Probabilistic Models [70.3497683558609]
  We present an approach for continual learning based on fully probabilistic (or generative) models of machine learning. We propose a pseudo-rehearsal approach using a Gaussian Mixture Model (GMM) instance for both generator and classifier functionalities. We show that GMR achieves state-of-the-art performance on common class-incremental learning problems at very competitive time and memory complexity.
  arXiv  Detail & Related papers  (2021-04-19T12:26:26Z)
• An empirical study of domain-agnostic semi-supervised learning via energy-based models: joint-training and pre-training [16.14838937433809]
  generative SSL methods involve unsupervised learning based on generative models by either joint-training or pre-training. Joint-training estimates the joint distribution of observations and labels, while pre-training is taken over observations only. It is found that joint-training EBMs outperform pre-training EBMs marginally but nearly consistently.
  arXiv  Detail & Related papers  (2020-10-25T13:35:23Z)
• Robust Finite Mixture Regression for Heterogeneous Targets [70.19798470463378]
  We propose an FMR model that finds sample clusters and jointly models multiple incomplete mixed-type targets simultaneously. We provide non-asymptotic oracle performance bounds for our model under a high-dimensional learning framework. The results show that our model can achieve state-of-the-art performance.
  arXiv  Detail & Related papers  (2020-10-12T03:27:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.