Graph Spectral Feature Learning for Mixed Data of Categorical and Numerical Type

• URL: http://arxiv.org/abs/2005.02817v1
• Date: Wed, 6 May 2020 13:36:59 GMT
• Title: Graph Spectral Feature Learning for Mixed Data of Categorical and Numerical Type
• Abstract summary: For simple neighborhood queries under mixed data space, standard practice is to consider numerical and categorical variables separately. We propose a novel strategy to explicitly model the probabilistic dependence structure among the mixed type of variables by an undirected graph.
• Score: 0.7106986689736827
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Feature learning in the presence of a mixed type of variables, numerical and categorical types, is an important issue for related modeling problems. For simple neighborhood queries under mixed data space, standard practice is to consider numerical and categorical variables separately and combining them based on some suitable distance functions. Alternatives, such as Kernel learning or Principal Component do not explicitly consider the inter-dependence structure among the mixed type of variables. In this work, we propose a novel strategy to explicitly model the probabilistic dependence structure among the mixed type of variables by an undirected graph. Spectral decomposition of the graph Laplacian provides the desired feature transformation. The Eigen spectrum of the transformed feature space shows increased separability and more prominent clusterability among the observations. The main novelty of our paper lies in capturing interactions of the mixed feature type in an unsupervised framework using a graphical model. We numerically validate the implications of the feature learning strategy

Related papers

• Profile Graphical Models [42.365931112630754]
  We introduce a novel class of graphical models, termed profile graphical models, that represent, within a single graph, how an external factor influences the dependence structure of a multivariate set of variables.<n>We derive key structural and probabilistic properties that underpin a more powerful inferential framework than existing approaches.<n>We show that the resulting profile undirected graphical models are independence-compatible with two-block LWF chain graph models.
  arXiv  Detail & Related papers  (2026-03-30T13:31:21Z)
• Improving Heterogeneous Graph Learning with Weighted Mixed-Curvature Product Manifold [4.640835690336652]
  In graph representation learning, the complex geometric structure of the input graph, e.g. hidden relations among nodes, is well captured in embedding space. Standard Euclidean embedding spaces have a limited capacity in representing graphs of varying structures. A promising candidate for the faithful embedding of data with varying structure is product manifold embedding spaces.
  arXiv  Detail & Related papers  (2023-07-10T12:20:50Z)
• High-Dimensional Undirected Graphical Models for Arbitrary Mixed Data [2.2871867623460207]
  In many applications data span variables of different types, whose principled joint analysis is nontrivial. Recent advances have shown how the binary-continuous case can be tackled, but the general mixed variable type regime remains challenging. We propose flexible and scalable methodology for data with variables of entirely general mixed type.
  arXiv  Detail & Related papers  (2022-11-21T18:21:31Z)
• Dynamic Latent Separation for Deep Learning [67.62190501599176]
  A core problem in machine learning is to learn expressive latent variables for model prediction on complex data. Here, we develop an approach that improves expressiveness, provides partial interpretation, and is not restricted to specific applications.
  arXiv  Detail & Related papers  (2022-10-07T17:56:53Z)
• On the Strong Correlation Between Model Invariance and Generalization [54.812786542023325]
  Generalization captures a model's ability to classify unseen data. Invariance measures consistency of model predictions on transformations of the data. From a dataset-centric view, we find a certain model's accuracy and invariance linearly correlated on different test sets.
  arXiv  Detail & Related papers  (2022-07-14T17:08:25Z)
• Linear Discriminant Analysis with High-dimensional Mixed Variables [10.774094462083843]
  This paper develops a novel approach for classifying high-dimensional observations with mixed variables. We overcome the challenge of having to split data into exponentially many cells. Results on the estimation accuracy and the misclassification rates are established.
  arXiv  Detail & Related papers  (2021-12-14T03:57:56Z)
• Semi-Supervised Disentanglement of Class-Related and Class-Independent Factors in VAE [4.533408938245526]
  We propose a framework capable of disentangling class-related and class-independent factors of variation in data. Our framework employs an attention mechanism in its latent space in order to improve the process of extracting class-related factors from data. Experiments show that our framework disentangles class-related and class-independent factors of variation and learns interpretable features.
  arXiv  Detail & Related papers  (2021-02-01T15:05:24Z)
• Hyperbolic Graph Embedding with Enhanced Semi-Implicit Variational Inference [48.63194907060615]
  We build off of semi-implicit graph variational auto-encoders to capture higher-order statistics in a low-dimensional graph latent representation. We incorporate hyperbolic geometry in the latent space through a Poincare embedding to efficiently represent graphs exhibiting hierarchical structure.
  arXiv  Detail & Related papers  (2020-10-31T05:48:34Z)
• Learning Representation for Mixed Data Types with a Nonlinear Deep Encoder-Decoder Framework [0.7106986689736827]
  We propose a novel nonlinear Deep-Decoder framework to capture the cross-domain information for mixed data types. We encode the information on the numerical variables in hidden nonlinear units. A separate and similar network is developed switching the roles of the numerical and categorical variables.
  arXiv  Detail & Related papers  (2020-09-21T06:29:49Z)
• Out-of-distribution Generalization via Partial Feature Decorrelation [72.96261704851683]
  We present a novel Partial Feature Decorrelation Learning (PFDL) algorithm, which jointly optimize a feature decomposition network and the target image classification model. The experiments on real-world datasets demonstrate that our method can improve the backbone model's accuracy on OOD image classification datasets.
  arXiv  Detail & Related papers  (2020-07-30T05:48:48Z)
• Structural Landmarking and Interaction Modelling: on Resolution Dilemmas in Graph Classification [50.83222170524406]
  We study the intrinsic difficulty in graph classification under the unified concept of resolution dilemmas'' We propose SLIM'', an inductive neural network model for Structural Landmarking and Interaction Modelling.
  arXiv  Detail & Related papers  (2020-06-29T01:01:42Z)
• Bayesian Sparse Factor Analysis with Kernelized Observations [67.60224656603823]
  Multi-view problems can be faced with latent variable models. High-dimensionality and non-linear issues are traditionally handled by kernel methods. We propose merging both approaches into single model.
  arXiv  Detail & Related papers  (2020-06-01T14:25:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.