Federated Learning for Short Text Clustering

• URL: http://arxiv.org/abs/2312.07556v1
• Date: Thu, 23 Nov 2023 12:19:41 GMT
• Title: Federated Learning for Short Text Clustering
• Authors: Mengling Hu, Chaochao Chen, Weiming Liu, Xinting Liao, and Xiaolin Zheng
• Abstract summary: We propose a Federated Robust Short Text Clustering (FSTC) framework for short text clustering. The robust short text clustering module aims to train an effective short text clustering model with local data in each client. The federated cluster center aggregation module aims to exchange knowledge across clients without sharing local raw data.
• Score: 21.308142639645517
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Short text clustering has been popularly studied for its significance in mining valuable insights from many short texts. In this paper, we focus on the federated short text clustering (FSTC) problem, i.e., clustering short texts that are distributed in different clients, which is a realistic problem under privacy requirements. Compared with the centralized short text clustering problem that short texts are stored on a central server, the FSTC problem has not been explored yet. To fill this gap, we propose a Federated Robust Short Text Clustering (FSTC) framework. FSTC includes two main modules, i.e., robust short text clustering module and federated cluster center aggregation module. The robust short text clustering module aims to train an effective short text clustering model with local data in each client. We innovatively combine optimal transport to generate pseudo-labels with Gaussian-uniform mixture model to ensure the reliability of the pseudo-supervised data. The federated cluster center aggregation module aims to exchange knowledge across clients without sharing local raw data in an efficient way. The server aggregates the local cluster centers from different clients and then sends the global centers back to all clients in each communication round. Our empirical studies on three short text clustering datasets demonstrate that FSTC significantly outperforms the federated short text clustering baselines.

Related papers

• Reliable Pseudo-labeling via Optimal Transport with Attention for Short Text Clustering [6.182375768528008]
  This paper proposes a novel short text clustering framework, called Reliable textbfPseudo-labeling via textbfOptimal textbfTransport. textbfPOTA generates reliable pseudo-labels to aid discriminative representation learning for clustering.
  arXiv  Detail & Related papers  (2025-01-25T12:13:38Z)
• CCFC: Bridging Federated Clustering and Contrastive Learning [9.91610928326645]
  We propose a new federated clustering method named cluster-contrastive federated clustering (CCFC) CCFC shows superior performance in handling device failures from a practical viewpoint.
  arXiv  Detail & Related papers  (2024-01-12T15:26:44Z)
• Dynamically Weighted Federated k-Means [0.0]
  Federated clustering enables multiple data sources to collaboratively cluster their data, maintaining decentralization and preserving privacy. We introduce a novel federated clustering algorithm named Dynamically Weighted Federated k-means (DWF k-means) based on Lloyd's method for k-means clustering. We conduct experiments on multiple datasets and data distribution settings to evaluate the performance of our algorithm in terms of clustering score, accuracy, and v-measure.
  arXiv  Detail & Related papers  (2023-10-23T12:28:21Z)
• Large Language Models Enable Few-Shot Clustering [88.06276828752553]
  We show that large language models can amplify an expert's guidance to enable query-efficient, few-shot semi-supervised text clustering. We find incorporating LLMs in the first two stages can routinely provide significant improvements in cluster quality.
  arXiv  Detail & Related papers  (2023-07-02T09:17:11Z)
• Timely Asynchronous Hierarchical Federated Learning: Age of Convergence [59.96266198512243]
  We consider an asynchronous hierarchical federated learning setting with a client-edge-cloud framework. The clients exchange the trained parameters with their corresponding edge servers, which update the locally aggregated model. The goal of each client is to converge to the global model, while maintaining timeliness of the clients.
  arXiv  Detail & Related papers  (2023-06-21T17:39:16Z)
• ClusterLLM: Large Language Models as a Guide for Text Clustering [45.835625439515]
  We introduce ClusterLLM, a novel text clustering framework that leverages feedback from an instruction-tuned large language model, such as ChatGPT. ClusterLLM consistently improves clustering quality, at an average cost of $0.6 per dataset.
  arXiv  Detail & Related papers  (2023-05-24T08:24:25Z)
• Hard Regularization to Prevent Deep Online Clustering Collapse without Data Augmentation [65.268245109828]
  Online deep clustering refers to the joint use of a feature extraction network and a clustering model to assign cluster labels to each new data point or batch as it is processed. While faster and more versatile than offline methods, online clustering can easily reach the collapsed solution where the encoder maps all inputs to the same point and all are put into a single cluster. We propose a method that does not require data augmentation, and that, differently from existing methods, regularizes the hard assignments.
  arXiv  Detail & Related papers  (2023-03-29T08:23:26Z)
• Optimizing Server-side Aggregation For Robust Federated Learning via Subspace Training [80.03567604524268]
  Non-IID data distribution across clients and poisoning attacks are two main challenges in real-world federated learning systems. We propose SmartFL, a generic approach that optimize the server-side aggregation process. We provide theoretical analyses of the convergence and generalization capacity for SmartFL.
  arXiv  Detail & Related papers  (2022-11-10T13:20:56Z)
• Efficient Distribution Similarity Identification in Clustered Federated Learning via Principal Angles Between Client Data Subspaces [59.33965805898736]
  Clustered learning has been shown to produce promising results by grouping clients into clusters. Existing FL algorithms are essentially trying to group clients together with similar distributions. Prior FL algorithms attempt similarities indirectly during training.
  arXiv  Detail & Related papers  (2022-09-21T17:37:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.