Graph-Convolutional Networks: Named Entity Recognition and Large Language Model Embedding in Document Clustering

• URL: http://arxiv.org/abs/2412.14867v1
• Date: Thu, 19 Dec 2024 14:03:22 GMT
• Title: Graph-Convolutional Networks: Named Entity Recognition and Large Language Model Embedding in Document Clustering
• Authors: Imed Keraghel, Mohamed Nadif,
• Abstract summary: This paper proposes a novel approach that integrates Named Entity Recognition (NER) and Large Language Models (LLMs) embeddings within a graph-based framework for document clustering. The method builds a graph with nodes representing documents and edges weighted by named entity similarity, optimized using a graph-convolutional network (GCN) Experimental results indicate that our approach outperforms conventional co-occurrence-based methods in clustering, notably for documents rich in named entities.
• Score: 9.929301228994095
• License:
• Abstract: Recent advances in machine learning, particularly Large Language Models (LLMs) such as BERT and GPT, provide rich contextual embeddings that improve text representation. However, current document clustering approaches often ignore the deeper relationships between named entities (NEs) and the potential of LLM embeddings. This paper proposes a novel approach that integrates Named Entity Recognition (NER) and LLM embeddings within a graph-based framework for document clustering. The method builds a graph with nodes representing documents and edges weighted by named entity similarity, optimized using a graph-convolutional network (GCN). This ensures a more effective grouping of semantically related documents. Experimental results indicate that our approach outperforms conventional co-occurrence-based methods in clustering, notably for documents rich in named entities.

Related papers

• Information-Theoretic Generative Clustering of Documents [24.56214029342293]
  We present generative clustering (GC) for clustering a set of documents, $mathrmX$. Because large language models (LLMs) provide probability distributions, the similarity between two documents can be rigorously defined. We show GC achieves the state-of-the-art performance, outperforming any previous clustering method often by a large margin.
  arXiv  Detail & Related papers  (2024-12-18T06:21:21Z)
• Knowledge-Aware Query Expansion with Large Language Models for Textual and Relational Retrieval [49.42043077545341]
  We propose a knowledge-aware query expansion framework, augmenting LLMs with structured document relations from knowledge graph (KG) We leverage document texts as rich KG node representations and use document-based relation filtering for our Knowledge-Aware Retrieval (KAR)
  arXiv  Detail & Related papers  (2024-10-17T17:03:23Z)
• Hypergraph based Understanding for Document Semantic Entity Recognition [65.84258776834524]
  We build a novel hypergraph attention document semantic entity recognition framework, HGA, which uses hypergraph attention to focus on entity boundaries and entity categories at the same time. Our results on FUNSD, CORD, XFUNDIE show that our method can effectively improve the performance of semantic entity recognition tasks.
  arXiv  Detail & Related papers  (2024-07-09T14:35:49Z)
• Unleashing the Potential of Text-attributed Graphs: Automatic Relation Decomposition via Large Language Models [31.443478448031886]
  RoSE (Relation-oriented Semantic Edge-decomposition) is a novel framework that decomposes the graph structure by analyzing raw text attributes. Our framework significantly enhances node classification performance across various datasets, with improvements of up to 16% on the Wisconsin dataset.
  arXiv  Detail & Related papers  (2024-05-28T20:54:47Z)
• Autoregressive Search Engines: Generating Substrings as Document Identifiers [53.0729058170278]
  Autoregressive language models are emerging as the de-facto standard for generating answers. Previous work has explored ways to partition the search space into hierarchical structures. In this work we propose an alternative that doesn't force any structure in the search space: using all ngrams in a passage as its possible identifiers.
  arXiv  Detail & Related papers  (2022-04-22T10:45:01Z)
• Attention-driven Graph Clustering Network [49.040136530379094]
  We propose a novel deep clustering method named Attention-driven Graph Clustering Network (AGCN) AGCN exploits a heterogeneous-wise fusion module to dynamically fuse the node attribute feature and the topological graph feature. AGCN can jointly perform feature learning and cluster assignment in an unsupervised fashion.
  arXiv  Detail & Related papers  (2021-08-12T02:30:38Z)
• Event-Driven News Stream Clustering using Entity-Aware Contextual Embeddings [14.225334321146779]
  We propose a method for online news stream clustering that is a variant of the non-parametric streaming K-means algorithm. Our model uses a combination of sparse and dense document representations, aggregates document-cluster similarity along these multiple representations. We show that the use of a suitable fine-tuning objective and external knowledge in pre-trained transformer models yields significant improvements in the effectiveness of contextual embeddings.
  arXiv  Detail & Related papers  (2021-01-26T19:58:30Z)
• Learning the Implicit Semantic Representation on Graph-Structured Data [57.670106959061634]
  Existing representation learning methods in graph convolutional networks are mainly designed by describing the neighborhood of each node as a perceptual whole. We propose a Semantic Graph Convolutional Networks (SGCN) that explores the implicit semantics by learning latent semantic-paths in graphs.
  arXiv  Detail & Related papers  (2021-01-16T16:18:43Z)
• Leveraging Graph to Improve Abstractive Multi-Document Summarization [50.62418656177642]
  We develop a neural abstractive multi-document summarization (MDS) model which can leverage well-known graph representations of documents. Our model utilizes graphs to encode documents in order to capture cross-document relations, which is crucial to summarizing long documents. Our model can also take advantage of graphs to guide the summary generation process, which is beneficial for generating coherent and concise summaries.
  arXiv  Detail & Related papers  (2020-05-20T13:39:47Z)
• Document Network Projection in Pretrained Word Embedding Space [7.455546102930911]
  We present Regularized Linear Embedding (RLE), a novel method that projects a collection of linked documents into a pretrained word embedding space. We leverage a matrix of pairwise similarities providing complementary information (e.g., the network proximity of two documents in a citation graph) The document representations can help to solve many information retrieval tasks, such as recommendation, classification and clustering.
  arXiv  Detail & Related papers  (2020-01-16T10:16:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.