Fair Clustering via Hierarchical Fair-Dirichlet Process

• URL: http://arxiv.org/abs/2305.17557v1
• Date: Sat, 27 May 2023 19:16:55 GMT
• Title: Fair Clustering via Hierarchical Fair-Dirichlet Process
• Authors: Abhisek Chakraborty, Anirban Bhattacharya, Debdeep Pati
• Abstract summary: A popular notion of fairness in clustering mandates the clusters to be em balanced, i.e., each level of a protected attribute must be approximately equally represented in each cluster. In this article, we offer a novel model-based formulation of fair clustering, complementing the existing literature which is almost exclusively based on optimizing appropriate objective functions.
• Score: 8.85031165304586
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: The advent of ML-driven decision-making and policy formation has led to an increasing focus on algorithmic fairness. As clustering is one of the most commonly used unsupervised machine learning approaches, there has naturally been a proliferation of literature on {\em fair clustering}. A popular notion of fairness in clustering mandates the clusters to be {\em balanced}, i.e., each level of a protected attribute must be approximately equally represented in each cluster. Building upon the original framework, this literature has rapidly expanded in various aspects. In this article, we offer a novel model-based formulation of fair clustering, complementing the existing literature which is almost exclusively based on optimizing appropriate objective functions.

Related papers

• GCC: Generative Calibration Clustering [55.44944397168619]
  We propose a novel Generative Clustering (GCC) method to incorporate feature learning and augmentation into clustering procedure. First, we develop a discrimirative feature alignment mechanism to discover intrinsic relationship across real and generated samples. Second, we design a self-supervised metric learning to generate more reliable cluster assignment.
  arXiv  Detail & Related papers  (2024-04-14T01:51:11Z)
• Large Language Models Enable Few-Shot Clustering [88.06276828752553]
  We show that large language models can amplify an expert's guidance to enable query-efficient, few-shot semi-supervised text clustering. We find incorporating LLMs in the first two stages can routinely provide significant improvements in cluster quality.
  arXiv  Detail & Related papers  (2023-07-02T09:17:11Z)
• Instance-Optimal Cluster Recovery in the Labeled Stochastic Block Model [79.46465138631592]
  We devise an efficient algorithm that recovers clusters using the observed labels. We present Instance-Adaptive Clustering (IAC), the first algorithm whose performance matches these lower bounds both in expectation and with high probability.
  arXiv  Detail & Related papers  (2023-06-18T08:46:06Z)
• CEIL: A General Classification-Enhanced Iterative Learning Framework for Text Clustering [16.08402937918212]
  We propose a novel Classification-Enhanced Iterative Learning framework for short text clustering. In each iteration, we first adopt a language model to retrieve the initial text representations. After strict data filtering and aggregation processes, samples with clean category labels are retrieved, which serve as supervision information. Finally, the updated language model with improved representation ability is used to enhance clustering in the next iteration.
  arXiv  Detail & Related papers  (2023-04-20T14:04:31Z)
• Cluster-level Group Representativity Fairness in $k$-means Clustering [3.420467786581458]
  Clustering algorithms could generate clusters such that different groups are disadvantaged within different clusters. We develop a clustering algorithm, building upon the centroid clustering paradigm pioneered by classical algorithms. We show that our method is effective in enhancing cluster-level group representativity fairness significantly at low impact on cluster coherence.
  arXiv  Detail & Related papers  (2022-12-29T22:02:28Z)
• Learning the Precise Feature for Cluster Assignment [39.320210567860485]
  We propose a framework which integrates representation learning and clustering into a single pipeline for the first time. The proposed framework exploits the powerful ability of recently developed generative models for learning intrinsic features. Experimental results show that the performance of the proposed method is superior, or at least comparable to, the state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-06-11T04:08:54Z)
• You Never Cluster Alone [150.94921340034688]
  We extend the mainstream contrastive learning paradigm to a cluster-level scheme, where all the data subjected to the same cluster contribute to a unified representation. We define a set of categorical variables as clustering assignment confidence, which links the instance-level learning track with the cluster-level one. By reparametrizing the assignment variables, TCC is trained end-to-end, requiring no alternating steps.
  arXiv  Detail & Related papers  (2021-06-03T14:59:59Z)
• Deep Fair Discriminative Clustering [24.237000220172906]
  We study a general notion of group-level fairness for binary and multi-state protected status variables (PSVs) We propose a refinement learning algorithm to combine the clustering goal with the fairness objective to learn fair clusters adaptively. Our framework shows promising results for novel clustering tasks including flexible fairness constraints, multi-state PSVs and predictive clustering.
  arXiv  Detail & Related papers  (2021-05-28T23:50:48Z)
• Scalable Hierarchical Agglomerative Clustering [65.66407726145619]
  Existing scalable hierarchical clustering methods sacrifice quality for speed. We present a scalable, agglomerative method for hierarchical clustering that does not sacrifice quality and scales to billions of data points.
  arXiv  Detail & Related papers  (2020-10-22T15:58:35Z)
• Fair Hierarchical Clustering [92.03780518164108]
  We define a notion of fairness that mitigates over-representation in traditional clustering. We show that our algorithms can find a fair hierarchical clustering, with only a negligible loss in the objective.
  arXiv  Detail & Related papers  (2020-06-18T01:05:11Z)
• Fair Algorithms for Hierarchical Agglomerative Clustering [17.66340013352806]
  Hierarchical Agglomerative Clustering (HAC) algorithms are extensively utilized in modern data science. It is imperative to ensure that these algorithms are fair -- even if the dataset contains biases against certain protected groups. We propose fair algorithms for performing HAC that enforce fairness constraints.
  arXiv  Detail & Related papers  (2020-05-07T01:41:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.