NILC: Discovering New Intents with LLM-assisted Clustering

• URL: http://arxiv.org/abs/2511.05913v1
• Date: Sat, 08 Nov 2025 08:18:44 GMT
• Title: NILC: Discovering New Intents with LLM-assisted Clustering
• Authors: Hongtao Wang, Renchi Yang, Wenqing Lin,
• Abstract summary: New intent discovery (NID) seeks to recognize both new and known intents from unlabeled user utterances.<n>This paper proposes NILC, a novel clustering framework specially catered for effective NID.
• Score: 15.077590298929719
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: New intent discovery (NID) seeks to recognize both new and known intents from unlabeled user utterances, which finds prevalent use in practical dialogue systems. Existing works towards NID mainly adopt a cascaded architecture, wherein the first stage focuses on encoding the utterances into informative text embeddings beforehand, while the latter is to group similar embeddings into clusters (i.e., intents), typically by K-Means. However, such a cascaded pipeline fails to leverage the feedback from both steps for mutual refinement, and, meanwhile, the embedding-only clustering overlooks nuanced textual semantics, leading to suboptimal performance. To bridge this gap, this paper proposes NILC, a novel clustering framework specially catered for effective NID. Particularly, NILC follows an iterative workflow, in which clustering assignments are judiciously updated by carefully refining cluster centroids and text embeddings of uncertain utterances with the aid of large language models (LLMs). Specifically, NILC first taps into LLMs to create additional semantic centroids for clusters, thereby enriching the contextual semantics of the Euclidean centroids of embeddings. Moreover, LLMs are then harnessed to augment hard samples (ambiguous or terse utterances) identified from clusters via rewriting for subsequent cluster correction. Further, we inject supervision signals through non-trivial techniques seeding and soft must links for more accurate NID in the semi-supervised setting. Extensive experiments comparing NILC against multiple recent baselines under both unsupervised and semi-supervised settings showcase that NILC can achieve significant performance improvements over six benchmark datasets of diverse domains consistently.

Related papers

• Fine-grained Semantics Integration for Large Language Model-based Recommendation [35.75224379727093]
  We propose TS-Rec, which can integrate Token-level Semantics into LLM-based Recommenders.<n>Extensive experiments on two real-world benchmarks demonstrate that TS-Rec consistently outperforms traditional and generative baselines.
  arXiv  Detail & Related papers  (2026-02-26T05:17:24Z)
• ClusterFusion: Hybrid Clustering with Embedding Guidance and LLM Adaptation [52.794544682493814]
  Large language models (LLMs) provide strong contextual reasoning, yet prior work mainly uses them as auxiliary modules to refine embeddings or adjust cluster boundaries.<n>We propose ClusterFusion, a hybrid framework that treats the LLM as the clustering core, guided by lightweight embedding methods.<n> Experiments on three public benchmarks and two new domain-specific datasets demonstrate that ClusterFusion achieves state-of-the-art performance on standard tasks.
  arXiv  Detail & Related papers  (2025-12-04T00:49:43Z)
• Text Clustering as Classification with LLMs [9.128151647718251]
  We propose a novel framework that reframes text clustering as a classification task by harnessing the in-context learning capabilities of Large Language Models.<n>By leveraging the advanced natural language understanding and generalization capabilities of LLMs, the proposed approach enables effective clustering with minimal human intervention.<n> Experimental results on diverse datasets demonstrate that our framework achieves comparable or superior performance to state-of-the-art embedding-based clustering techniques.
  arXiv  Detail & Related papers  (2024-09-30T16:57:34Z)
• Unleash LLMs Potential for Recommendation by Coordinating Twin-Tower Dynamic Semantic Token Generator [60.07198935747619]
  We propose Twin-Tower Dynamic Semantic Recommender (T TDS), the first generative RS which adopts dynamic semantic index paradigm. To be more specific, we for the first time contrive a dynamic knowledge fusion framework which integrates a twin-tower semantic token generator into the LLM-based recommender. The proposed T TDS recommender achieves an average improvement of 19.41% in Hit-Rate and 20.84% in NDCG metric, compared with the leading baseline methods.
  arXiv  Detail & Related papers  (2024-09-14T01:45:04Z)
• OMH: Structured Sparsity via Optimally Matched Hierarchy for Unsupervised Semantic Segmentation [69.37484603556307]
  Un Semantic segmenting (USS) involves segmenting images without relying on predefined labels. We introduce a novel approach called Optimally Matched Hierarchy (OMH) to simultaneously address the above issues. Our OMH yields better unsupervised segmentation performance compared to existing USS methods.
  arXiv  Detail & Related papers  (2024-03-11T09:46:41Z)
• End-to-end Learnable Clustering for Intent Learning in Recommendation [54.157784572994316]
  We propose a novel intent learning method termed underlineELCRec. It unifies behavior representation learning into an underlineEnd-to-end underlineLearnable underlineClustering framework. We deploy this method on the industrial recommendation system with 130 million page views and achieve promising results.
  arXiv  Detail & Related papers  (2024-01-11T15:22:55Z)
• Large Language Models Enable Few-Shot Clustering [88.06276828752553]
  We show that large language models can amplify an expert's guidance to enable query-efficient, few-shot semi-supervised text clustering. We find incorporating LLMs in the first two stages can routinely provide significant improvements in cluster quality.
  arXiv  Detail & Related papers  (2023-07-02T09:17:11Z)
• Scalable Learning of Latent Language Structure With Logical Offline Cycle Consistency [71.42261918225773]
  Conceptually, LOCCO can be viewed as a form of self-learning where the semantic being trained is used to generate annotations for unlabeled text. As an added bonus, the annotations produced by LOCCO can be trivially repurposed to train a neural text generation model.
  arXiv  Detail & Related papers  (2023-05-31T16:47:20Z)
• CEIL: A General Classification-Enhanced Iterative Learning Framework for Text Clustering [16.08402937918212]
  We propose a novel Classification-Enhanced Iterative Learning framework for short text clustering. In each iteration, we first adopt a language model to retrieve the initial text representations. After strict data filtering and aggregation processes, samples with clean category labels are retrieved, which serve as supervision information. Finally, the updated language model with improved representation ability is used to enhance clustering in the next iteration.
  arXiv  Detail & Related papers  (2023-04-20T14:04:31Z)
• A Clustering Framework for Unsupervised and Semi-supervised New Intent Discovery [25.900661912504397]
  We propose a novel clustering framework, USNID, for unsupervised and semi-supervised new intent discovery. First, it fully utilizes unsupervised or semi-supervised data to mine shallow semantic similarity relations. Second, it designs a centroid-guided clustering mechanism to address the issue of cluster allocation inconsistency. Third, it captures high-level semantics in unsupervised or semi-supervised data to discover fine-grained intent-wise clusters.
  arXiv  Detail & Related papers  (2023-04-16T05:30:42Z)
• A Proposition-Level Clustering Approach for Multi-Document Summarization [82.4616498914049]
  We revisit the clustering approach, grouping together propositions for more precise information alignment. Our method detects salient propositions, clusters them into paraphrastic clusters, and generates a representative sentence for each cluster by fusing its propositions. Our summarization method improves over the previous state-of-the-art MDS method in the DUC 2004 and TAC 2011 datasets.
  arXiv  Detail & Related papers  (2021-12-16T10:34:22Z)
• A Classification-Based Approach to Semi-Supervised Clustering with Pairwise Constraints [5.639904484784126]
  We introduce a network framework for semi-supervised clustering with pairwise constraints. In contrast to existing approaches, we decompose SSC into two simpler classification tasks/stages. The proposed approach, S3C2, is motivated by the observation that binary classification is usually easier than multi-class clustering.
  arXiv  Detail & Related papers  (2020-01-18T20:13:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.