Meta Clustering for Collaborative Learning
- URL: http://arxiv.org/abs/2006.00082v3
- Date: Tue, 27 Sep 2022 20:54:49 GMT
- Title: Meta Clustering for Collaborative Learning
- Authors: Chenglong Ye, Reza Ghanadan, Jie Ding
- Abstract summary: In collaborative learning, learners coordinate to enhance each of their learning performances.
From the perspective of any learner, a critical challenge is to filter out unqualified collaborators.
We propose a framework named meta clustering to address the challenge.
- Score: 13.003650251457193
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In collaborative learning, learners coordinate to enhance each of their
learning performances. From the perspective of any learner, a critical
challenge is to filter out unqualified collaborators. We propose a framework
named meta clustering to address the challenge. Unlike the classical problem of
clustering data points, meta clustering categorizes learners. Assuming each
learner performs a supervised regression on a standalone local dataset, we
propose a Select-Exchange-Cluster (SEC) method to classify the learners by
their underlying supervised functions. We theoretically show that the SEC can
cluster learners into accurate collaboration sets. Empirical studies
corroborate the theoretical analysis and demonstrate that SEC can be
computationally efficient, robust against learner heterogeneity, and effective
in enhancing single-learner performance. Also, we show how the proposed
approach may be used to enhance data fairness. Supplementary materials for this
article are available online.
Related papers
- Data Similarity-Based One-Shot Clustering for Multi-Task Hierarchical Federated Learning [8.37314799155978]
We propose a one-shot clustering algorithm that can effectively identify and group users based on their data similarity.
Our proposed algorithm not only enhances the clustering process, but also overcomes challenges related to privacy concerns, communication overhead, and the need for prior knowledge about learning models or loss function behaviors.
arXiv Detail & Related papers (2024-10-03T17:51:21Z) - Discriminative Anchor Learning for Efficient Multi-view Clustering [59.11406089896875]
We propose discriminative anchor learning for multi-view clustering (DALMC)
We learn discriminative view-specific feature representations according to the original dataset.
We build anchors from different views based on these representations, which increase the quality of the shared anchor graph.
arXiv Detail & Related papers (2024-09-25T13:11:17Z) - GraphLearner: Graph Node Clustering with Fully Learnable Augmentation [76.63963385662426]
Contrastive deep graph clustering (CDGC) leverages the power of contrastive learning to group nodes into different clusters.
We propose a Graph Node Clustering with Fully Learnable Augmentation, termed GraphLearner.
It introduces learnable augmentors to generate high-quality and task-specific augmented samples for CDGC.
arXiv Detail & Related papers (2022-12-07T10:19:39Z) - Using Representation Expressiveness and Learnability to Evaluate
Self-Supervised Learning Methods [61.49061000562676]
We introduce Cluster Learnability (CL) to assess learnability.
CL is measured in terms of the performance of a KNN trained to predict labels obtained by clustering the representations with K-means.
We find that CL better correlates with in-distribution model performance than other competing recent evaluation schemes.
arXiv Detail & Related papers (2022-06-02T19:05:13Z) - Integrating Auxiliary Information in Self-supervised Learning [94.11964997622435]
We first observe that the auxiliary information may bring us useful information about data structures.
We present to construct data clusters according to the auxiliary information.
We show that Cl-InfoNCE may be a better approach to leverage the data clustering information.
arXiv Detail & Related papers (2021-06-05T11:01:15Z) - You Never Cluster Alone [150.94921340034688]
We extend the mainstream contrastive learning paradigm to a cluster-level scheme, where all the data subjected to the same cluster contribute to a unified representation.
We define a set of categorical variables as clustering assignment confidence, which links the instance-level learning track with the cluster-level one.
By reparametrizing the assignment variables, TCC is trained end-to-end, requiring no alternating steps.
arXiv Detail & Related papers (2021-06-03T14:59:59Z) - Deep Fair Discriminative Clustering [24.237000220172906]
We study a general notion of group-level fairness for binary and multi-state protected status variables (PSVs)
We propose a refinement learning algorithm to combine the clustering goal with the fairness objective to learn fair clusters adaptively.
Our framework shows promising results for novel clustering tasks including flexible fairness constraints, multi-state PSVs and predictive clustering.
arXiv Detail & Related papers (2021-05-28T23:50:48Z) - Unsupervised Visual Representation Learning by Online Constrained
K-Means [44.38989920488318]
Cluster discrimination is an effective pretext task for unsupervised representation learning.
We propose a novel clustering-based pretext task with online textbfConstrained textbfK-mtextbfeans (textbfCoKe)
Our online assignment method has a theoretical guarantee to approach the global optimum.
arXiv Detail & Related papers (2021-05-24T20:38:32Z) - Clustering based Contrastive Learning for Improving Face Representations [34.75646290505793]
We present Clustering-based Contrastive Learning (CCL), a new clustering-based representation learning approach.
CCL uses labels obtained from clustering along with video constraints to learn discnative face features.
arXiv Detail & Related papers (2020-04-05T13:11:44Z) - Mining Implicit Entity Preference from User-Item Interaction Data for
Knowledge Graph Completion via Adversarial Learning [82.46332224556257]
We propose a novel adversarial learning approach by leveraging user interaction data for the Knowledge Graph Completion task.
Our generator is isolated from user interaction data, and serves to improve the performance of the discriminator.
To discover implicit entity preference of users, we design an elaborate collaborative learning algorithms based on graph neural networks.
arXiv Detail & Related papers (2020-03-28T05:47:33Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.