Alternatives of Unsupervised Representations of Variables on the Latent Space

• URL: http://arxiv.org/abs/2410.20172v1
• Date: Sat, 26 Oct 2024 13:06:35 GMT
• Title: Alternatives of Unsupervised Representations of Variables on the Latent Space
• Authors: Alex Glushkovsky,
• Abstract summary: The article addresses the application of unsupervised machine learning to represent variables on the 2D latent space by applying a variational autoencoder (beta-VAE) Five distinct methods have been introduced to represent variables on the latent space. Twenty-eight approaches of variable representations by beta-VAE have been considered.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The article addresses the application of unsupervised machine learning to represent variables on the 2D latent space by applying a variational autoencoder (beta-VAE). Representation of variables on low dimensional spaces allows for data visualization, disentanglement of variables based on underlying characteristics, finding of meaningful patterns and outliers, and supports interpretability. Five distinct methods have been introduced to represent variables on the latent space: (1) straightforward transposed, (2) univariate metadata of variables, such as variable statistics, empirical probability density and cumulative distribution functions, (3) adjacency matrices of different metrics, such as correlations, R2 values, Jaccard index, cosine similarity, and mutual information, (4) gradient mappings followed by spot cross product calculation, and (5) combined. Twenty-eight approaches of variable representations by beta-VAE have been considered. The pairwise spot cross product addresses relationships of gradients of two variables along latent space axes, such as orthogonal, confounded positive, confounded negative, and everything in between. The article addresses generalized representations of variables that cover both features and labels. Dealing with categorical variables, reinforced entanglement has been introduced to represent one-hot encoded categories. The article includes three examples: (1) synthetic data with known dependencies, (2) famous MNIST example of handwritten numbers, and (3) real-world multivariate time series of Canadian financial market interest rates. As a result, unsupervised representations of interest rates on the latent space correctly disentangled rates based on their type, such as bonds, T-bills, GICs, or conventional mortgages, positioned bonds and T-bills along a single curve, and ordered rates by their terms along that curve.

Related papers

• Better Together: Cross and Joint Covariances Enhance Signal Detectability in Undersampled Data [0.0]
  Many data-science applications involve detecting a shared signal between two high-dimensional variables.<n>Using random matrix theory methods, we determine when such signal can be detected and reconstructed from sample correlations.<n>We discuss what these observations mean for choosing the right method for detecting linear correlations in data and how these findings may generalize to nonlinear statistical dependencies.
  arXiv  Detail & Related papers  (2025-07-29T20:11:02Z)
• How Do Transformers Learn Variable Binding in Symbolic Programs? [5.611678524375841]
  We train a Transformer to dereference queried variables in symbolic programs.<n>We find that the model learns to exploit the residual stream as an addressable memory space.<n>Our results show how Transformer models can learn to implement systematic variable binding.
  arXiv  Detail & Related papers  (2025-05-27T08:39:20Z)
• Disentanglement Analysis in Deep Latent Variable Models Matching Aggregate Posterior Distributions [0.5759862457142761]
  We propose a method to evaluate disentanglement for deep latent variable models (DLVMs) in general. The proposed technique discovers the latent vectors representing the generative factors of a dataset that can be different from the cardinal latent axes.
  arXiv  Detail & Related papers  (2025-01-26T23:38:39Z)
• Unsupervised Representation Learning from Sparse Transformation Analysis [79.94858534887801]
  We propose to learn representations from sequence data by factorizing the transformations of the latent variables into sparse components. Input data are first encoded as distributions of latent activations and subsequently transformed using a probability flow model.
  arXiv  Detail & Related papers  (2024-10-07T23:53:25Z)
• CAVIAR: Categorical-Variable Embeddings for Accurate and Robust Inference [0.2209921757303168]
  Social science research often hinges on the relationship between categorical variables and outcomes. We introduce CAVIAR, a novel method for embedding categorical variables that assume values in a high-dimensional ambient space but are sampled from an underlying manifold.
  arXiv  Detail & Related papers  (2024-04-07T14:47:07Z)
• Robust Concept Erasure via Kernelized Rate-Distortion Maximization [38.19696482602788]
  We propose a new distance metric learning-based objective, the Kernelized Rate-Distortion Maximizer (KRaM) KRaM fits a transformation of representations to match a specified distance measure (defined by a labeled concept to erase) using a modified rate-distortion function. We find that KRaM effectively erases various types of concepts: categorical, continuous, and vector-valued variables from data representations across diverse domains.
  arXiv  Detail & Related papers  (2023-11-30T21:10:44Z)
• Identifiable Latent Causal Content for Domain Adaptation under Latent Covariate Shift [82.14087963690561]
  Multi-source domain adaptation (MSDA) addresses the challenge of learning a label prediction function for an unlabeled target domain. We present an intricate causal generative model by introducing latent noises across domains, along with a latent content variable and a latent style variable. The proposed approach showcases exceptional performance and efficacy on both simulated and real-world datasets.
  arXiv  Detail & Related papers  (2022-08-30T11:25:15Z)
• On the Strong Correlation Between Model Invariance and Generalization [54.812786542023325]
  Generalization captures a model's ability to classify unseen data. Invariance measures consistency of model predictions on transformations of the data. From a dataset-centric view, we find a certain model's accuracy and invariance linearly correlated on different test sets.
  arXiv  Detail & Related papers  (2022-07-14T17:08:25Z)
• Linear Discriminant Analysis with High-dimensional Mixed Variables [10.774094462083843]
  This paper develops a novel approach for classifying high-dimensional observations with mixed variables. We overcome the challenge of having to split data into exponentially many cells. Results on the estimation accuracy and the misclassification rates are established.
  arXiv  Detail & Related papers  (2021-12-14T03:57:56Z)
• Learning Conditional Invariance through Cycle Consistency [60.85059977904014]
  We propose a novel approach to identify meaningful and independent factors of variation in a dataset. Our method involves two separate latent subspaces for the target property and the remaining input information. We demonstrate on synthetic and molecular data that our approach identifies more meaningful factors which lead to sparser and more interpretable models.
  arXiv  Detail & Related papers  (2021-11-25T17:33:12Z)
• Triplot: model agnostic measures and visualisations for variable importance in predictive models that take into account the hierarchical correlation structure [3.0036519884678894]
  We propose new methods to support model analysis by exploiting the information about the correlation between variables. We show how to analyze groups of variables (aspects) both when they are proposed by the user and when they should be determined automatically. We also present the new type of model visualisation, triplot, which exploits a hierarchical structure of variable grouping to produce a high information density model visualisation.
  arXiv  Detail & Related papers  (2021-04-07T21:29:03Z)
• Tractable Inference in Credal Sentential Decision Diagrams [116.6516175350871]
  Probabilistic sentential decision diagrams are logic circuits where the inputs of disjunctive gates are annotated by probability values. We develop the credal sentential decision diagrams, a generalisation of their probabilistic counterpart that allows for replacing the local probabilities with credal sets of mass functions. For a first empirical validation, we consider a simple application based on noisy seven-segment display images.
  arXiv  Detail & Related papers  (2020-08-19T16:04:34Z)
• Learning Disentangled Representations with Latent Variation Predictability [102.4163768995288]
  This paper defines the variation predictability of latent disentangled representations. Within an adversarial generation process, we encourage variation predictability by maximizing the mutual information between latent variations and corresponding image pairs. We develop an evaluation metric that does not rely on the ground-truth generative factors to measure the disentanglement of latent representations.
  arXiv  Detail & Related papers  (2020-07-25T08:54:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.