On the Use of Relative Validity Indices for Comparing Clustering Approaches

• URL: http://arxiv.org/abs/2404.10351v2
• Date: Thu, 21 Nov 2024 00:57:22 GMT
• Title: On the Use of Relative Validity Indices for Comparing Clustering Approaches
• Authors: Luke W. Yerbury, Ricardo J. G. B. Campello, G. C. Livingston Jr, Mark Goldsworthy, Lachlan O'Neil,
• Abstract summary: Relative Validity Indices are widely used for evaluating and optimising clustering outcomes. There is a growing trend in the literature to use RVIs when selecting a Similarity Paradigm (SP) for clustering. This study presents the first comprehensive investigation into the reliability of RVIs for SP selection.
• Score: 0.6990493129893111
• License:
• Abstract: Relative Validity Indices (RVIs) such as the Silhouette Width Criterion and Davies Bouldin indices are the most widely used tools for evaluating and optimising clustering outcomes. Traditionally, their ability to rank collections of candidate dataset partitions has been used to guide the selection of the number of clusters, and to compare partitions from different clustering algorithms. However, there is a growing trend in the literature to use RVIs when selecting a Similarity Paradigm (SP) for clustering - the combination of normalisation procedure, representation method, and distance measure which affects the computation of object dissimilarities used in clustering. Despite the growing prevalence of this practice, there has been no empirical or theoretical investigation into the suitability of RVIs for this purpose. Moreover, since RVIs are computed using object dissimilarities, it remains unclear how they would need to be implemented for fair comparisons of different SPs. This study presents the first comprehensive investigation into the reliability of RVIs for SP selection. We conducted extensive experiments with seven popular RVIs on over 2.7 million clustering partitions of synthetic and real-world datasets, encompassing feature-vector and time-series data. We identified fundamental conceptual limitations undermining the use of RVIs for SP selection, and our empirical findings confirmed this predicted unsuitability. Among our recommendations, we suggest instead that practitioners select SPs by using external validation on high quality labelled datasets or carefully designed outcome-oriented objective criteria, both of which should be informed by careful consideration of dataset characteristics, and domain requirements. Our findings have important implications for clustering methodology and evaluation, suggesting the need for more rigorous approaches to SP selection.

Related papers

• A Bayesian cluster validity index [0.0]
  Cluster validity indices (CVIs) are designed to identify the optimal number of clusters within a dataset. We introduce a Bayesian cluster validity index (BCVI) which builds upon existing indices. Our BCVI offers clear advantages in situations where user expertise is valuable, allowing users to specify their desired range for the final number of clusters.
  arXiv  Detail & Related papers  (2024-02-03T14:23:36Z)
• Differentially Private Federated Clustering over Non-IID Data [59.611244450530315]
  clustering clusters (FedC) problem aims to accurately partition unlabeled data samples distributed over massive clients into finite clients under the orchestration of a server. We propose a novel FedC algorithm using differential privacy convergence technique, referred to as DP-Fed, in which partial participation and multiple clients are also considered. Various attributes of the proposed DP-Fed are obtained through theoretical analyses of privacy protection, especially for the case of non-identically and independently distributed (non-i.i.d.) data.
  arXiv  Detail & Related papers  (2023-01-03T05:38:43Z)
• Robust Consensus Clustering and its Applications for Advertising Forecasting [18.242055675730253]
  We propose a novel algorithm -- robust consensus clustering that can find common ground truth among experts' opinions. We apply the proposed method to the real-world advertising campaign segmentation and forecasting tasks.
  arXiv  Detail & Related papers  (2022-12-27T21:49:04Z)
• A One-shot Framework for Distributed Clustered Learning in Heterogeneous Environments [54.172993875654015]
  The paper proposes a family of communication efficient methods for distributed learning in heterogeneous environments. One-shot approach, based on local computations at the users and a clustering based aggregation step at the server is shown to provide strong learning guarantees. For strongly convex problems it is shown that, as long as the number of data points per user is above a threshold, the proposed approach achieves order-optimal mean-squared error rates in terms of the sample size.
  arXiv  Detail & Related papers  (2022-09-22T09:04:10Z)
• Impact of Load Demand Dataset Characteristics on Clustering Validation Indices [1.5749416770494706]
  Clustering households based on their demand profiles is one of the primary, yet a key component of such analysis. Various cluster validation indices (CVIs) have been proposed in the literature. This paper shows how the recommendations of validation indices are influenced by different data characteristics.
  arXiv  Detail & Related papers  (2021-08-03T12:22:34Z)
• Doing Great at Estimating CATE? On the Neglected Assumptions in Benchmark Comparisons of Treatment Effect Estimators [91.3755431537592]
  We show that even in arguably the simplest setting, estimation under ignorability assumptions can be misleading. We consider two popular machine learning benchmark datasets for evaluation of heterogeneous treatment effect estimators. We highlight that the inherent characteristics of the benchmark datasets favor some algorithms over others.
  arXiv  Detail & Related papers  (2021-07-28T13:21:27Z)
• A Distance-based Separability Measure for Internal Cluster Validation [0.0]
  Internal cluster validity indices (CVIs) are used to evaluate clustering results in unsupervised learning. We propose Distance-based Separability Index (DSI) based on a data separability measure. Results show DSI is an effective, unique, and competitive CVI to other compared CVIs.
  arXiv  Detail & Related papers  (2021-06-17T20:19:50Z)
• Combining Task Predictors via Enhancing Joint Predictability [53.46348489300652]
  We present a new predictor combination algorithm that improves the target by i) measuring the relevance of references based on their capabilities in predicting the target, and ii) strengthening such estimated relevance. Our algorithm jointly assesses the relevance of all references by adopting a Bayesian framework. Based on experiments on seven real-world datasets from visual attribute ranking and multi-class classification scenarios, we demonstrate that our algorithm offers a significant performance gain and broadens the application range of existing predictor combination approaches.
  arXiv  Detail & Related papers  (2020-07-15T21:58:39Z)
• Decorrelated Clustering with Data Selection Bias [55.91842043124102]
  We propose a novel Decorrelation regularized K-Means algorithm (DCKM) for clustering with data selection bias. Our DCKM algorithm achieves significant performance gains, indicating the necessity of removing unexpected feature correlations induced by selection bias.
  arXiv  Detail & Related papers  (2020-06-29T08:55:50Z)
• Clustering Binary Data by Application of Combinatorial Optimization Heuristics [52.77024349608834]
  We study clustering methods for binary data, first defining aggregation criteria that measure the compactness of clusters. Five new and original methods are introduced, using neighborhoods and population behavior optimization metaheuristics. From a set of 16 data tables generated by a quasi-Monte Carlo experiment, a comparison is performed for one of the aggregations using L1 dissimilarity, with hierarchical clustering, and a version of k-means: partitioning around medoids or PAM.
  arXiv  Detail & Related papers  (2020-01-06T23:33:31Z)
• On clustering uncertain and structured data with Wasserstein barycenters and a geodesic criterion for the number of clusters [0.0]
  This work considers the notion of Wasserstein barycenters, accompanied by appropriate clustering indices based on the intrinsic geometry of the Wasserstein space where the clustering task is performed. Such type of clustering approaches are highly appreciated in many fields where the observational/experimental error is significant. Under this perspective, each observation is identified by an appropriate probability measure and the proposed clustering schemes rely on discrimination criteria.
  arXiv  Detail & Related papers  (2019-12-26T08:46:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.