An upper bound of the silhouette validation metric for clustering

• URL: http://arxiv.org/abs/2509.08625v2
• Date: Sun, 09 Nov 2025 02:23:29 GMT
• Title: An upper bound of the silhouette validation metric for clustering
• Abstract summary: The average silhouette width (ASW) is a widely used internal measure of clustering quality.<n>In this work, we derive for each data point a sharp upper bound on its silhouette width and aggregate these to obtain a canonical upper bound on the ASW.<n>This bound-often substantially below 1-enhances the interpretability of empirical ASW values by indicating how close a given clustering result is to the best possible outcome on that dataset.
• Score: 0.552480439325792
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The silhouette coefficient quantifies, for each observation, the balance between within-cluster cohesion and between-cluster separation, taking values in [-1, 1]. The average silhouette width (ASW) is a widely used internal measure of clustering quality, with higher values indicating more cohesive and well-separated clusters. However, the dataset-specific maximum of ASW is typically unknown, and the standard upper limit of 1 is rarely attainable. In this work, we derive for each data point a sharp upper bound on its silhouette width and aggregate these to obtain a canonical upper bound on the ASW. This bound-often substantially below 1-enhances the interpretability of empirical ASW values by indicating how close a given clustering result is to the best possible outcome on that dataset. It can be used to confirm global optimality, guide the evaluation of clustering solutions, and be refined to incorporate minimum cluster-size constraints for greater practical relevance. Finally, we extend the framework to establish a corresponding bound for the macro-averaged silhouette.

Related papers

• Composite Silhouette: A Subsampling-based Aggregation Strategy [5.70259485914933]
  We introduce Composite Silhouette, an internal criterion for cluster-count selection.<n>It aggregates evidence across repeated subsampled clusterings rather than relying on a single partition.<n>Experiments on synthetic and real-world datasets show that Composite Silhouette effectively reconciles the strengths of micro- and macro-averaging.
  arXiv  Detail & Related papers  (2026-04-15T12:49:09Z)
• The generalized underlap coefficient with an application in clustering [2.140305355990306]
  Underlap coefficient (UNL) is a multi-group separation measure.<n>We establish key properties of the UNL and provide an explicit connection to total variation.<n>We illustrate the application of the UNL in clustering using two real world datasets.
  arXiv  Detail & Related papers  (2026-02-23T03:35:08Z)
• Towards Learnable Anchor for Deep Multi-View Clustering [49.767879678193005]
  In this paper, we propose the Deep Multi-view Anchor Clustering (DMAC) model that performs clustering in linear time.<n>With the optimal anchors, the full sample graph is calculated to derive a discriminative embedding for clustering.<n>Experiments on several datasets demonstrate superior performance and efficiency of DMAC compared to state-of-the-art competitors.
  arXiv  Detail & Related papers  (2025-03-16T09:38:11Z)
• Village-Net Clustering: A Rapid approach to Non-linear Unsupervised Clustering of High-Dimensional Data [0.0]
  We develop an unsupervised clustering algorithm, we call "Village-Net"<n>The algorithm operates in two phases: first, utilizing K-Means clustering, it divides the dataset into distinct subsets we refer to as "villages"<n>We present extensive benchmarking on extant real-world datasets with known ground-truth labels to showcase its competitive performance.
  arXiv  Detail & Related papers  (2025-01-16T06:56:43Z)
• Similarity and Dissimilarity Guided Co-association Matrix Construction for Ensemble Clustering [22.280221709474105]
  We propose the Similarity and Dissimilarity Guided Co-association matrix (SDGCA) to achieve ensemble clustering. First, we introduce normalized ensemble entropy to estimate the quality of each cluster, and construct a similarity matrix based on this estimation. We employ the random walk to explore high-order proximity of base clusterings to construct a dissimilarity matrix.
  arXiv  Detail & Related papers  (2024-11-01T08:10:28Z)
• Self-Supervised Graph Embedding Clustering [70.36328717683297]
  K-means one-step dimensionality reduction clustering method has made some progress in addressing the curse of dimensionality in clustering tasks. We propose a unified framework that integrates manifold learning with K-means, resulting in the self-supervised graph embedding framework.
  arXiv  Detail & Related papers  (2024-09-24T08:59:51Z)
• A Framework for Fine-Tuning LLMs using Heterogeneous Feedback [69.51729152929413]
  We present a framework for fine-tuning large language models (LLMs) using heterogeneous feedback. First, we combine the heterogeneous feedback data into a single supervision format, compatible with methods like SFT and RLHF. Next, given this unified feedback dataset, we extract a high-quality and diverse subset to obtain performance increases.
  arXiv  Detail & Related papers  (2024-08-05T23:20:32Z)
• Deep Clustering Using the Soft Silhouette Score: Towards Compact and Well-Separated Clusters [0.0]
  We propose soft silhoutte, a probabilistic formulation of the silhouette coefficient. We introduce an autoencoder-based deep learning architecture that is suitable for optimizing the soft silhouette objective function. The proposed deep clustering method has been tested and compared with several well-studied deep clustering methods on various benchmark datasets.
  arXiv  Detail & Related papers  (2024-02-01T14:02:06Z)
• Rethinking k-means from manifold learning perspective [122.38667613245151]
  We present a new clustering algorithm which directly detects clusters of data without mean estimation. Specifically, we construct distance matrix between data points by Butterworth filter. To well exploit the complementary information embedded in different views, we leverage the tensor Schatten p-norm regularization.
  arXiv  Detail & Related papers  (2023-05-12T03:01:41Z)
• A Generalized Framework for Predictive Clustering and Optimization [18.06697544912383]
  Clustering is a powerful and extensively used data science tool. In this article, we define a generalized optimization framework for predictive clustering. We also present a joint optimization strategy that exploits mixed-integer linear programming (MILP) for global optimization.
  arXiv  Detail & Related papers  (2023-05-07T19:56:51Z)
• Multi-View Clustering via Semi-non-negative Tensor Factorization [120.87318230985653]
  We develop a novel multi-view clustering based on semi-non-negative tensor factorization (Semi-NTF) Our model directly considers the between-view relationship and exploits the between-view complementary information. In addition, we provide an optimization algorithm for the proposed method and prove mathematically that the algorithm always converges to the stationary KKT point.
  arXiv  Detail & Related papers  (2023-03-29T14:54:19Z)
• Shape complexity in cluster analysis [0.0]
  In cluster analysis, a common first step is to scale the data aiming to better partition them into clusters. Here we explore the use of multidimensional shapes of data, aiming to obtain scaling factors for use prior to clustering. We give results on some iconic data sets, highlighting the strengths and potential weaknesses of the new approach.
  arXiv  Detail & Related papers  (2022-05-17T01:33:15Z)
• Anomaly Clustering: Grouping Images into Coherent Clusters of Anomaly Types [60.45942774425782]
  We introduce anomaly clustering, whose goal is to group data into coherent clusters of anomaly types. This is different from anomaly detection, whose goal is to divide anomalies from normal data. We present a simple yet effective clustering framework using a patch-based pretrained deep embeddings and off-the-shelf clustering methods.
  arXiv  Detail & Related papers  (2021-12-21T23:11:33Z)
• Does the Data Induce Capacity Control in Deep Learning? [0.0]
  This paper studies how the dataset may be the cause of the anomalous generalization performance of deep networks. We show that the data correlation matrix of typical classification datasets has an eigenspectrum where, after a sharp initial drop, a large number of small eigenvalues are distributed uniformly over an exponentially large range.
  arXiv  Detail & Related papers  (2021-10-27T04:40:27Z)
• Local versions of sum-of-norms clustering [77.34726150561087]
  We show that our method can separate arbitrarily close balls in the ball model. We prove a quantitative bound on the error incurred in the clustering of disjoint connected sets.
  arXiv  Detail & Related papers  (2021-09-20T14:45:29Z)
• Scalable Hierarchical Agglomerative Clustering [65.66407726145619]
  Existing scalable hierarchical clustering methods sacrifice quality for speed. We present a scalable, agglomerative method for hierarchical clustering that does not sacrifice quality and scales to billions of data points.
  arXiv  Detail & Related papers  (2020-10-22T15:58:35Z)
• Contrastive Clustering [57.71729650297379]
  We propose Contrastive Clustering (CC) which explicitly performs the instance- and cluster-level contrastive learning. In particular, CC achieves an NMI of 0.705 (0.431) on the CIFAR-10 (CIFAR-100) dataset, which is an up to 19% (39%) performance improvement compared with the best baseline.
  arXiv  Detail & Related papers  (2020-09-21T08:54:40Z)
• Clustering small datasets in high-dimension by random projection [2.2940141855172027]
  We propose a low-computation method to find statistically significant clustering structures in a small dataset. The method proceeds by projecting the data on a random line and seeking binary clusterings in the resulting one-dimensional data. The statistical validity of the clustering structures obtained is tested in the projected one-dimensional space.
  arXiv  Detail & Related papers  (2020-08-21T16:49:37Z)
• LSD-C: Linearly Separable Deep Clusters [145.89790963544314]
  We present LSD-C, a novel method to identify clusters in an unlabeled dataset. Our method draws inspiration from recent semi-supervised learning practice and proposes to combine our clustering algorithm with self-supervised pretraining and strong data augmentation. We show that our approach significantly outperforms competitors on popular public image benchmarks including CIFAR 10/100, STL 10 and MNIST, as well as the document classification dataset Reuters 10K.
  arXiv  Detail & Related papers  (2020-06-17T17:58:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.