Sparse and geometry-aware generalisation of the mutual information for joint discriminative clustering and feature selection

• URL: http://arxiv.org/abs/2302.03391v2
• Date: Thu, 18 Jul 2024 09:44:18 GMT
• Title: Sparse and geometry-aware generalisation of the mutual information for joint discriminative clustering and feature selection
• Authors: Louis Ohl, Pierre-Alexandre Mattei, Charles Bouveyron, Mickaël Leclercq, Arnaud Droit, Frédéric Precioso,
• Abstract summary: We introduce a discriminative clustering model trying to maximise a geometry-aware generalisation of the mutual information called GEMINI. This algorithm avoids the burden of feature exploration and is easily scalable to high-dimensional data and large amounts of samples while only designing a discriminative clustering model. Our results show that Sparse GEMINI is a competitive algorithm and has the ability to select relevant subsets of variables with respect to the clustering without using relevance criteria or prior hypotheses.
• Score: 19.066989850964756
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Feature selection in clustering is a hard task which involves simultaneously the discovery of relevant clusters as well as relevant variables with respect to these clusters. While feature selection algorithms are often model-based through optimised model selection or strong assumptions on the data distribution, we introduce a discriminative clustering model trying to maximise a geometry-aware generalisation of the mutual information called GEMINI with a simple l1 penalty: the Sparse GEMINI. This algorithm avoids the burden of combinatorial feature subset exploration and is easily scalable to high-dimensional data and large amounts of samples while only designing a discriminative clustering model. We demonstrate the performances of Sparse GEMINI on synthetic datasets and large-scale datasets. Our results show that Sparse GEMINI is a competitive algorithm and has the ability to select relevant subsets of variables with respect to the clustering without using relevance criteria or prior hypotheses.

Related papers

• Instance-Optimal Cluster Recovery in the Labeled Stochastic Block Model [79.46465138631592]
  We devise an efficient algorithm that recovers clusters using the observed labels. We present Instance-Adaptive Clustering (IAC), the first algorithm whose performance matches these lower bounds both in expectation and with high probability.
  arXiv  Detail & Related papers  (2023-06-18T08:46:06Z)
• High-dimensional variable clustering based on maxima of a weakly dependent random process [1.1999555634662633]
  We propose a new class of models for variable clustering called Asymptotic Independent block (AI-block) models. This class of models is identifiable, meaning that there exists a maximal element with a partial order between partitions, allowing for statistical inference. We also present an algorithm depending on a tuning parameter that recovers the clusters of variables without specifying the number of clusters empha priori.
  arXiv  Detail & Related papers  (2023-02-02T08:24:26Z)
• Unified Multi-View Orthonormal Non-Negative Graph Based Clustering Framework [74.25493157757943]
  We formulate a novel clustering model, which exploits the non-negative feature property and incorporates the multi-view information into a unified joint learning framework. We also explore, for the first time, the multi-model non-negative graph-based approach to clustering data based on deep features.
  arXiv  Detail & Related papers  (2022-11-03T08:18:27Z)
• Clustering Optimisation Method for Highly Connected Biological Data [0.0]
  We show how a simple metric for connectivity clustering evaluation leads to an optimised segmentation of biological data. The novelty of the work resides in the creation of a simple optimisation method for clustering crowded data.
  arXiv  Detail & Related papers  (2022-08-08T17:33:32Z)
• Personalized Federated Learning via Convex Clustering [72.15857783681658]
  We propose a family of algorithms for personalized federated learning with locally convex user costs. The proposed framework is based on a generalization of convex clustering in which the differences between different users' models are penalized.
  arXiv  Detail & Related papers  (2022-02-01T19:25:31Z)
• Differentially-Private Clustering of Easy Instances [67.04951703461657]
  In differentially private clustering, the goal is to identify $k$ cluster centers without disclosing information on individual data points. We provide implementable differentially private clustering algorithms that provide utility when the data is "easy" We propose a framework that allows us to apply non-private clustering algorithms to the easy instances and privately combine the results.
  arXiv  Detail & Related papers  (2021-12-29T08:13:56Z)
• Clustering-Based Subset Selection in Evolutionary Multiobjective Optimization [11.110675371854988]
  Subset selection is an important component in evolutionary multiobjective optimization (EMO) algorithms. Clustering-based methods have not been evaluated in the context of subset selection from solution sets obtained by EMO algorithms.
  arXiv  Detail & Related papers  (2021-08-19T02:56:41Z)
• Scalable Hierarchical Agglomerative Clustering [65.66407726145619]
  Existing scalable hierarchical clustering methods sacrifice quality for speed. We present a scalable, agglomerative method for hierarchical clustering that does not sacrifice quality and scales to billions of data points.
  arXiv  Detail & Related papers  (2020-10-22T15:58:35Z)
• Mixed data Deep Gaussian Mixture Model: A clustering model for mixed datasets [0.0]
  We introduce a model-based clustering method called Mixed Deep Gaussian Mixture Model (MDGMM) This architecture is flexible and can be adapted to mixed as well as to continuous or non-continuous data. Our model provides continuous low-dimensional representations of the data which can be a useful tool to visualize mixed datasets.
  arXiv  Detail & Related papers  (2020-10-13T19:52:46Z)
• EGMM: an Evidential Version of the Gaussian Mixture Model for Clustering [22.586481334904793]
  We propose a new model-based clustering algorithm, called EGMM (evidential GMM), in the theoretical framework of belief functions. The parameters in EGMM are estimated by a specially designed Expectation-Maximization (EM) algorithm. The proposed EGMM is as simple as the classical GMM, but can generate a more informative evidential partition for the considered dataset.
  arXiv  Detail & Related papers  (2020-10-03T11:59:07Z)
• Clustering Binary Data by Application of Combinatorial Optimization Heuristics [52.77024349608834]
  We study clustering methods for binary data, first defining aggregation criteria that measure the compactness of clusters. Five new and original methods are introduced, using neighborhoods and population behavior optimization metaheuristics. From a set of 16 data tables generated by a quasi-Monte Carlo experiment, a comparison is performed for one of the aggregations using L1 dissimilarity, with hierarchical clustering, and a version of k-means: partitioning around medoids or PAM.
  arXiv  Detail & Related papers  (2020-01-06T23:33:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.