Unsupervised hierarchical clustering using the learning dynamics of RBMs

• URL: http://arxiv.org/abs/2302.01851v3
• Date: Fri, 9 Jun 2023 17:05:09 GMT
• Title: Unsupervised hierarchical clustering using the learning dynamics of RBMs
• Authors: Aur\'elien Decelle, Lorenzo Rosset, Beatriz Seoane
• Abstract summary: We present a new and general method for building relational data trees by exploiting the learning dynamics of the Restricted Boltzmann Machine (RBM) Our method is based on the mean-field approach, derived from the Plefka expansion, and developed in context of disordered systems. We tested our method in an artificially hierarchical dataset and on three different real-world datasets (images of digits, mutations in the human genome, and a family of proteins)
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Datasets in the real world are often complex and to some degree hierarchical, with groups and sub-groups of data sharing common characteristics at different levels of abstraction. Understanding and uncovering the hidden structure of these datasets is an important task that has many practical applications. To address this challenge, we present a new and general method for building relational data trees by exploiting the learning dynamics of the Restricted Boltzmann Machine (RBM). Our method is based on the mean-field approach, derived from the Plefka expansion, and developed in the context of disordered systems. It is designed to be easily interpretable. We tested our method in an artificially created hierarchical dataset and on three different real-world datasets (images of digits, mutations in the human genome, and a homologous family of proteins). The method is able to automatically identify the hierarchical structure of the data. This could be useful in the study of homologous protein sequences, where the relationships between proteins are critical for understanding their function and evolution.

Related papers

• How compositional generalization and creativity improve as diffusion models are trained [82.08869888944324]
  How many samples do generative models need to learn the composition rules, so as to produce a number of novel data? We consider diffusion models trained on simple context-free grammars - tree-like graphical models used to represent the structure of data such as language and images. We demonstrate that diffusion models learn compositional rules with the sample complexity required for clustering features with statistically similar context, a process similar to the word2vec.
  arXiv  Detail & Related papers  (2025-02-17T18:06:33Z)
• Semantically Rich Local Dataset Generation for Explainable AI in Genomics [0.716879432974126]
  Black box deep learning models trained on genomic sequences excel at predicting the outcomes of different gene regulatory mechanisms. We propose using Genetic Programming to generate datasets by evolving perturbations in sequences that contribute to their semantic diversity.
  arXiv  Detail & Related papers  (2024-07-03T10:31:30Z)
• UniCell: Universal Cell Nucleus Classification via Prompt Learning [76.11864242047074]
  We propose a universal cell nucleus classification framework (UniCell) It employs a novel prompt learning mechanism to uniformly predict the corresponding categories of pathological images from different dataset domains. In particular, our framework adopts an end-to-end architecture for nuclei detection and classification, and utilizes flexible prediction heads for adapting various datasets.
  arXiv  Detail & Related papers  (2024-02-20T11:50:27Z)
• Homological Convolutional Neural Networks [4.615338063719135]
  We propose a novel deep learning architecture that exploits the data structural organization through topologically constrained network representations. We test our model on 18 benchmark datasets against 5 classic machine learning and 3 deep learning models.
  arXiv  Detail & Related papers  (2023-08-26T08:48:51Z)
• Hierarchical clustering with dot products recovers hidden tree structure [53.68551192799585]
  In this paper we offer a new perspective on the well established agglomerative clustering algorithm, focusing on recovery of hierarchical structure. We recommend a simple variant of the standard algorithm, in which clusters are merged by maximum average dot product and not, for example, by minimum distance or within-cluster variance. We demonstrate that the tree output by this algorithm provides a bona fide estimate of generative hierarchical structure in data, under a generic probabilistic graphical model.
  arXiv  Detail & Related papers  (2023-05-24T11:05:12Z)
• Topological Learning in Multi-Class Data Sets [0.3050152425444477]
  We study the impact of topological complexity on learning in feedforward deep neural networks (DNNs) We evaluate our topological classification algorithm on multiple constructed and open source data sets.
  arXiv  Detail & Related papers  (2023-01-23T21:54:25Z)
• Amortized Inference for Causal Structure Learning [72.84105256353801]
  Learning causal structure poses a search problem that typically involves evaluating structures using a score or independence test. We train a variational inference model to predict the causal structure from observational/interventional data. Our models exhibit robust generalization capabilities under substantial distribution shift.
  arXiv  Detail & Related papers  (2022-05-25T17:37:08Z)
• Joint Geometric and Topological Analysis of Hierarchical Datasets [7.098759778181621]
  In this paper, we focus on high-dimensional data that are organized into several hierarchical datasets. The main novelty in this work lies in the combination of two powerful data-analytic approaches: topological data analysis and geometric manifold learning. We show that our new method gives rise to superior classification results compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-04-03T13:02:00Z)
• Relation-Guided Representation Learning [53.60351496449232]
  We propose a new representation learning method that explicitly models and leverages sample relations. Our framework well preserves the relations between samples. By seeking to embed samples into subspace, we show that our method can address the large-scale and out-of-sample problem.
  arXiv  Detail & Related papers  (2020-07-11T10:57:45Z)
• A Trainable Optimal Transport Embedding for Feature Aggregation and its Relationship to Attention [96.77554122595578]
  We introduce a parametrized representation of fixed size, which embeds and then aggregates elements from a given input set according to the optimal transport plan between the set and a trainable reference. Our approach scales to large datasets and allows end-to-end training of the reference, while also providing a simple unsupervised learning mechanism with small computational cost.
  arXiv  Detail & Related papers  (2020-06-22T08:35:58Z)
• Knowledge Elicitation using Deep Metric Learning and Psychometric Testing [15.989397781243225]
  We provide a method for efficient hierarchical knowledge elicitation from experts working with high-dimensional data such as images or videos. The developed models embed the high-dimensional data in a metric space where distances are semantically meaningful, and the data can be organized in a hierarchical structure.
  arXiv  Detail & Related papers  (2020-04-14T08:33:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.