Meta Clustering for Collaborative Learning
- URL: http://arxiv.org/abs/2006.00082v3
- Date: Tue, 27 Sep 2022 20:54:49 GMT
- Title: Meta Clustering for Collaborative Learning
- Authors: Chenglong Ye, Reza Ghanadan, Jie Ding
- Abstract summary: In collaborative learning, learners coordinate to enhance each of their learning performances.
From the perspective of any learner, a critical challenge is to filter out unqualified collaborators.
We propose a framework named meta clustering to address the challenge.
- Score: 13.003650251457193
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In collaborative learning, learners coordinate to enhance each of their
learning performances. From the perspective of any learner, a critical
challenge is to filter out unqualified collaborators. We propose a framework
named meta clustering to address the challenge. Unlike the classical problem of
clustering data points, meta clustering categorizes learners. Assuming each
learner performs a supervised regression on a standalone local dataset, we
propose a Select-Exchange-Cluster (SEC) method to classify the learners by
their underlying supervised functions. We theoretically show that the SEC can
cluster learners into accurate collaboration sets. Empirical studies
corroborate the theoretical analysis and demonstrate that SEC can be
computationally efficient, robust against learner heterogeneity, and effective
in enhancing single-learner performance. Also, we show how the proposed
approach may be used to enhance data fairness. Supplementary materials for this
article are available online.
Related papers
- End-to-end Learnable Clustering for Intent Learning in Recommendation [61.29127008174193]
We propose a novel intent learning method termed underlineELCRec.
It unifies behavior representation learning into an underlineEnd-to-end underlineLearnable underlineClustering framework.
We deploy this method on the industrial recommendation system with 130 million page views and achieve promising results.
arXiv Detail & Related papers (2024-01-11T15:22:55Z) - Hard Regularization to Prevent Deep Online Clustering Collapse without
Data Augmentation [65.268245109828]
Online deep clustering refers to the joint use of a feature extraction network and a clustering model to assign cluster labels to each new data point or batch as it is processed.
While faster and more versatile than offline methods, online clustering can easily reach the collapsed solution where the encoder maps all inputs to the same point and all are put into a single cluster.
We propose a method that does not require data augmentation, and that, differently from existing methods, regularizes the hard assignments.
arXiv Detail & Related papers (2023-03-29T08:23:26Z) - Using Representation Expressiveness and Learnability to Evaluate
Self-Supervised Learning Methods [61.49061000562676]
We introduce Cluster Learnability (CL) to assess learnability.
CL is measured in terms of the performance of a KNN trained to predict labels obtained by clustering the representations with K-means.
We find that CL better correlates with in-distribution model performance than other competing recent evaluation schemes.
arXiv Detail & Related papers (2022-06-02T19:05:13Z) - Personalized Federated Learning with Multiple Known Clusters [20.585114235701603]
We consider the problem of personalized federated learning when there are known cluster structures within users.
An intuitive approach would be to regularize the parameters so that users in the same cluster share similar model weights.
We develop an algorithm that allows each cluster to communicate independently and derive the convergence results.
arXiv Detail & Related papers (2022-04-28T16:32:29Z) - Integrating Auxiliary Information in Self-supervised Learning [94.11964997622435]
We first observe that the auxiliary information may bring us useful information about data structures.
We present to construct data clusters according to the auxiliary information.
We show that Cl-InfoNCE may be a better approach to leverage the data clustering information.
arXiv Detail & Related papers (2021-06-05T11:01:15Z) - You Never Cluster Alone [150.94921340034688]
We extend the mainstream contrastive learning paradigm to a cluster-level scheme, where all the data subjected to the same cluster contribute to a unified representation.
We define a set of categorical variables as clustering assignment confidence, which links the instance-level learning track with the cluster-level one.
By reparametrizing the assignment variables, TCC is trained end-to-end, requiring no alternating steps.
arXiv Detail & Related papers (2021-06-03T14:59:59Z) - Deep Fair Discriminative Clustering [24.237000220172906]
We study a general notion of group-level fairness for binary and multi-state protected status variables (PSVs)
We propose a refinement learning algorithm to combine the clustering goal with the fairness objective to learn fair clusters adaptively.
Our framework shows promising results for novel clustering tasks including flexible fairness constraints, multi-state PSVs and predictive clustering.
arXiv Detail & Related papers (2021-05-28T23:50:48Z) - Unsupervised Visual Representation Learning by Online Constrained
K-Means [44.38989920488318]
Cluster discrimination is an effective pretext task for unsupervised representation learning.
We propose a novel clustering-based pretext task with online textbfConstrained textbfK-mtextbfeans (textbfCoKe)
Our online assignment method has a theoretical guarantee to approach the global optimum.
arXiv Detail & Related papers (2021-05-24T20:38:32Z) - Clustering based Contrastive Learning for Improving Face Representations [34.75646290505793]
We present Clustering-based Contrastive Learning (CCL), a new clustering-based representation learning approach.
CCL uses labels obtained from clustering along with video constraints to learn discnative face features.
arXiv Detail & Related papers (2020-04-05T13:11:44Z) - Mining Implicit Entity Preference from User-Item Interaction Data for
Knowledge Graph Completion via Adversarial Learning [82.46332224556257]
We propose a novel adversarial learning approach by leveraging user interaction data for the Knowledge Graph Completion task.
Our generator is isolated from user interaction data, and serves to improve the performance of the discriminator.
To discover implicit entity preference of users, we design an elaborate collaborative learning algorithms based on graph neural networks.
arXiv Detail & Related papers (2020-03-28T05:47:33Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.