Semi-supervised Learning by Latent Space Energy-Based Model of Symbol-Vector Coupling

• URL: http://arxiv.org/abs/2010.09359v1
• Date: Mon, 19 Oct 2020 09:55:14 GMT
• Title: Semi-supervised Learning by Latent Space Energy-Based Model of Symbol-Vector Coupling
• Authors: Bo Pang, Erik Nijkamp, Jiali Cui, Tian Han, Ying Nian Wu
• Abstract summary: We propose a latent space energy-based prior model for semi-supervised learning. We show that our method performs well on semi-supervised learning tasks.
• Score: 55.866810975092115
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper proposes a latent space energy-based prior model for semi-supervised learning. The model stands on a generator network that maps a latent vector to the observed example. The energy term of the prior model couples the latent vector and a symbolic one-hot vector, so that classification can be based on the latent vector inferred from the observed example. In our learning method, the symbol-vector coupling, the generator network and the inference network are learned jointly. Our method is applicable to semi-supervised learning in various data domains such as image, text, and tabular data. Our experiments demonstrate that our method performs well on semi-supervised learning tasks.

Related papers

• An Intrinsic Vector Heat Network [64.55434397799728]
  This paper introduces a novel neural network architecture for learning tangent vector fields embedded in 3D. We introduce a trainable vector heat diffusion module to spatially propagate vector-valued feature data across the surface. We also demonstrate the effectiveness of our method on the useful industrial application of quadrilateral mesh generation.
  arXiv  Detail & Related papers  (2024-06-14T00:40:31Z)
• Automatic Discovery of Visual Circuits [66.99553804855931]
  We explore scalable methods for extracting the subgraph of a vision model's computational graph that underlies recognition of a specific visual concept. We find that our approach extracts circuits that causally affect model output, and that editing these circuits can defend large pretrained models from adversarial attacks.
  arXiv  Detail & Related papers  (2024-04-22T17:00:57Z)
• Learning Sparse Latent Representations for Generator Model [7.467412443287767]
  We present a new unsupervised learning method to enforce sparsity on the latent space for the generator model. Our model consists of only one top-down generator network that maps the latent variable to the observed data.
  arXiv  Detail & Related papers  (2022-09-20T18:58:24Z)
• Latent Space Energy-Based Model of Symbol-Vector Coupling for Text Generation and Classification [42.5461221309397]
  We propose a latent space energy-based prior model for text generation and classification. The model stands on a generator network that generates the text sequence based on a continuous latent vector. Our experiments demonstrate that the proposed model learns well-structured and meaningful latent space.
  arXiv  Detail & Related papers  (2021-08-26T02:31:18Z)
• Asymptotics of Network Embeddings Learned via Subsampling [4.23373349945751]
  We study representation methods using a subsampling approach, such as node2vec, into a single unifying framework. This provides a theoretical foundation to understand what the embedding vectors represent and how well these methods perform on downstream tasks. Notably, we observe that typically used loss functions may lead to shortcomings, such as a lack of Fisher consistency.
  arXiv  Detail & Related papers  (2021-07-06T02:54:53Z)
• Prototypical Representation Learning for Relation Extraction [56.501332067073065]
  This paper aims to learn predictive, interpretable, and robust relation representations from distantly-labeled data. We learn prototypes for each relation from contextual information to best explore the intrinsic semantics of relations. Results on several relation learning tasks show that our model significantly outperforms the previous state-of-the-art relational models.
  arXiv  Detail & Related papers  (2021-03-22T08:11:43Z)
• word2vec, node2vec, graph2vec, X2vec: Towards a Theory of Vector Embeddings of Structured Data [2.63067287928779]
  We propose two theoretical approaches for understanding the foundations of vector embeddings. We draw connections between the various approaches and suggest directions for future research.
  arXiv  Detail & Related papers  (2020-03-27T18:23:55Z)
• Embedding Graph Auto-Encoder for Graph Clustering [90.8576971748142]
  Graph auto-encoder (GAE) models are based on semi-supervised graph convolution networks (GCN) We design a specific GAE-based model for graph clustering to be consistent with the theory, namely Embedding Graph Auto-Encoder (EGAE) EGAE consists of one encoder and dual decoders.
  arXiv  Detail & Related papers  (2020-02-20T09:53:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.