Reliable Pseudo-labeling via Optimal Transport with Attention for Short Text Clustering

• URL: http://arxiv.org/abs/2501.15194v3
• Date: Tue, 04 Feb 2025 06:44:31 GMT
• Title: Reliable Pseudo-labeling via Optimal Transport with Attention for Short Text Clustering
• Authors: Zhihao Yao, Jixuan Yin, Bo Li,
• Abstract summary: This paper proposes a novel short text clustering framework, called Reliable textbfPseudo-labeling via textbfOptimal textbfTransport. textbfPOTA generates reliable pseudo-labels to aid discriminative representation learning for clustering.
• Score: 6.182375768528008
• License:
• Abstract: Short text clustering has gained significant attention in the data mining community. However, the limited valuable information contained in short texts often leads to low-discriminative representations, increasing the difficulty of clustering. This paper proposes a novel short text clustering framework, called Reliable \textbf{P}seudo-labeling via \textbf{O}ptimal \textbf{T}ransport with \textbf{A}ttention for Short Text Clustering (\textbf{POTA}), that generate reliable pseudo-labels to aid discriminative representation learning for clustering. Specially, \textbf{POTA} first implements an instance-level attention mechanism to capture the semantic relationships among samples, which are then incorporated as a semantic consistency regularization term into an optimal transport problem. By solving this OT problem, we can yield reliable pseudo-labels that simultaneously account for sample-to-sample semantic consistency and sample-to-cluster global structure information. Additionally, the proposed OT can adaptively estimate cluster distributions, making \textbf{POTA} well-suited for varying degrees of imbalanced datasets. Then, we utilize the pseudo-labels to guide contrastive learning to generate discriminative representations and achieve efficient clustering. Extensive experiments demonstrate \textbf{POTA} outperforms state-of-the-art methods. The code is available at: \href{https://github.com/YZH0905/POTA-STC/tree/main}{https://github.com/YZH0905/POTA-STC/tree/main}.

Related papers

• k-LLMmeans: Summaries as Centroids for Interpretable and Scalable LLM-Based Text Clustering [0.0]
  We introduce k-LLMmeans, a novel modification of the k-means clustering algorithm that utilizes LLMs to generate textual summaries as cluster centroids. This modification preserves the properties of k-means while offering greater interpretability. We present a case study showcasing the interpretability of evolving cluster centroids in sequential text streams.
  arXiv  Detail & Related papers  (2025-02-12T19:50:22Z)
• Discriminative Representation learning via Attention-Enhanced Contrastive Learning for Short Text Clustering [1.6788443047694643]
  We propose a novel short text clustering method, called Discriminative Representation learning via textbfAttention-textbfEnhanced textbfContrastive textbfL. Experimental results demonstrate that the proposed textbfAECL outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-01-07T07:17:04Z)
• Self-Supervised Graph Embedding Clustering [70.36328717683297]
  K-means one-step dimensionality reduction clustering method has made some progress in addressing the curse of dimensionality in clustering tasks. We propose a unified framework that integrates manifold learning with K-means, resulting in the self-supervised graph embedding framework.
  arXiv  Detail & Related papers  (2024-09-24T08:59:51Z)
• Semantic Equitable Clustering: A Simple and Effective Strategy for Clustering Vision Tokens [57.37893387775829]
  We introduce a fast and balanced clustering method, named textbfSemantic textbfEquitable textbfClustering (SEC) SEC clusters tokens based on their global semantic relevance in an efficient, straightforward manner. We propose a versatile vision backbone, SECViT, to serve as a vision language connector.
  arXiv  Detail & Related papers  (2024-05-22T04:49:00Z)
• Federated Learning for Short Text Clustering [21.308142639645517]
  We propose a Federated Robust Short Text Clustering (FSTC) framework for short text clustering. The robust short text clustering module aims to train an effective short text clustering model with local data in each client. The federated cluster center aggregation module aims to exchange knowledge across clients without sharing local raw data.
  arXiv  Detail & Related papers  (2023-11-23T12:19:41Z)
• Generalized Category Discovery with Clustering Assignment Consistency [56.92546133591019]
  Generalized category discovery (GCD) is a recently proposed open-world task. We propose a co-training-based framework that encourages clustering consistency. Our method achieves state-of-the-art performance on three generic benchmarks and three fine-grained visual recognition datasets.
  arXiv  Detail & Related papers  (2023-10-30T00:32:47Z)
• Implicit Sample Extension for Unsupervised Person Re-Identification [97.46045935897608]
  Clustering sometimes mixes different true identities together or splits the same identity into two or more sub clusters. We propose an Implicit Sample Extension (OurWholeMethod) method to generate what we call support samples around the cluster boundaries. Experiments demonstrate that the proposed method is effective and achieves state-of-the-art performance for unsupervised person Re-ID.
  arXiv  Detail & Related papers  (2022-04-14T11:41:48Z)
• You Never Cluster Alone [150.94921340034688]
  We extend the mainstream contrastive learning paradigm to a cluster-level scheme, where all the data subjected to the same cluster contribute to a unified representation. We define a set of categorical variables as clustering assignment confidence, which links the instance-level learning track with the cluster-level one. By reparametrizing the assignment variables, TCC is trained end-to-end, requiring no alternating steps.
  arXiv  Detail & Related papers  (2021-06-03T14:59:59Z)
• Relation Clustering in Narrative Knowledge Graphs [71.98234178455398]
  relational sentences in the original text are embedded (with SBERT) and clustered in order to merge together semantically similar relations. Preliminary tests show that such clustering might successfully detect similar relations, and provide a valuable preprocessing for semi-supervised approaches.
  arXiv  Detail & Related papers  (2020-11-27T10:43:04Z)
• Enhancement of Short Text Clustering by Iterative Classification [0.0]
  iterative classification applies outlier removal to obtain outlier-free clusters. It trains a classification algorithm using the non-outliers based on their cluster distributions. By repeating this several times, we obtain a much improved clustering of texts.
  arXiv  Detail & Related papers  (2020-01-31T02:12:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.