Text Classification Based on Knowledge Graphs and Improved Attention Mechanism

• URL: http://arxiv.org/abs/2401.03591v2
• Date: Sat, 27 Jan 2024 00:01:26 GMT
• Title: Text Classification Based on Knowledge Graphs and Improved Attention Mechanism
• Authors: Siyu Li, Lu Chen, Chenwei Song, Xinyi Liu
• Abstract summary: The model operates at both character and word levels to deepen its understanding by integrating the concepts. Its performance is demonstrated on datasets such as AGNews, Ohsumed, and TagMyNews, achieving accuracy of 75.1%, 58.7%, and 68.5% respectively.
• Score: 12.008192698720947
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: To resolve the semantic ambiguity in texts, we propose a model, which innovatively combines a knowledge graph with an improved attention mechanism. An existing knowledge base is utilized to enrich the text with relevant contextual concepts. The model operates at both character and word levels to deepen its understanding by integrating the concepts. We first adopt information gain to select import words. Then an encoder-decoder framework is used to encode the text along with the related concepts. The local attention mechanism adjusts the weight of each concept, reducing the influence of irrelevant or noisy concepts during classification. We improve the calculation formula for attention scores in the local self-attention mechanism, ensuring that words with different frequencies of occurrence in the text receive higher attention scores. Finally, the model employs a Bi-directional Gated Recurrent Unit (Bi-GRU), which is effective in feature extraction from texts for improved classification accuracy. Its performance is demonstrated on datasets such as AGNews, Ohsumed, and TagMyNews, achieving accuracy of 75.1%, 58.7%, and 68.5% respectively, showing its effectiveness in classifying tasks.

Related papers

• A Novel Ehanced Move Recognition Algorithm Based on Pre-trained Models with Positional Embeddings [6.688643243555054]
  The recognition of abstracts is crucial for effectively locating the content and clarifying the article. This paper proposes a novel enhanced move recognition algorithm with an improved pre-trained model and a gated network with attention mechanism for unstructured abstracts of Chinese scientific and technological papers.
  arXiv  Detail & Related papers  (2023-08-14T03:20:28Z)
• Select and Augment: Enhanced Dense Retrieval Knowledge Graph Augmentation [0.59829224684009]
  We propose a framework that jointly selects a set of text descriptions relevant to KG entities as well as align or augment KG embeddings with text descriptions. Experiment results for Link Prediction demonstrate a 5.5% and 3.5% percentage increase in the Mean Reciprocal Rank (MRR) and Hits@10 scores respectively.
  arXiv  Detail & Related papers  (2023-07-28T19:33:18Z)
• Description-Enhanced Label Embedding Contrastive Learning for Text Classification [65.01077813330559]
  Self-Supervised Learning (SSL) in model learning process and design a novel self-supervised Relation of Relation (R2) classification task. Relation of Relation Learning Network (R2-Net) for text classification, in which text classification and R2 classification are treated as optimization targets. external knowledge from WordNet to obtain multi-aspect descriptions for label semantic learning.
  arXiv  Detail & Related papers  (2023-06-15T02:19:34Z)
• ChatGraph: Interpretable Text Classification by Converting ChatGPT Knowledge to Graphs [54.48467003509595]
  ChatGPT has shown superior performance in various natural language processing (NLP) tasks. We propose a novel framework that leverages the power of ChatGPT for specific tasks, such as text classification. Our method provides a more transparent decision-making process compared with previous text classification methods.
  arXiv  Detail & Related papers  (2023-05-03T19:57:43Z)
• Structure-Augmented Text Representation Learning for Efficient Knowledge Graph Completion [53.31911669146451]
  Human-curated knowledge graphs provide critical supportive information to various natural language processing tasks. These graphs are usually incomplete, urging auto-completion of them. graph embedding approaches, e.g., TransE, learn structured knowledge via representing graph elements into dense embeddings. textual encoding approaches, e.g., KG-BERT, resort to graph triple's text and triple-level contextualized representations.
  arXiv  Detail & Related papers  (2020-04-30T13:50:34Z)
• Exploiting Structured Knowledge in Text via Graph-Guided Representation Learning [73.0598186896953]
  We present two self-supervised tasks learning over raw text with the guidance from knowledge graphs. Building upon entity-level masked language models, our first contribution is an entity masking scheme. In contrast to existing paradigms, our approach uses knowledge graphs implicitly, only during pre-training.
  arXiv  Detail & Related papers  (2020-04-29T14:22:42Z)
• Salience Estimation with Multi-Attention Learning for Abstractive Text Summarization [86.45110800123216]
  In the task of text summarization, salience estimation for words, phrases or sentences is a critical component. We propose a Multi-Attention Learning framework which contains two new attention learning components for salience estimation.
  arXiv  Detail & Related papers  (2020-04-07T02:38:56Z)
• Short Text Classification via Knowledge powered Attention with Similarity Matrix based CNN [6.6723692875904375]
  We propose a knowledge powered attention with similarity matrix based convolutional neural network (KASM) model. We use knowledge graph (KG) to enrich the semantic representation of short text, specially, the information of parent-entity is introduced in our model. For the purpose of measuring the importance of knowledge, we introduce the attention mechanisms to choose the important information.
  arXiv  Detail & Related papers  (2020-02-09T12:08:43Z)
• Generative Adversarial Zero-Shot Relational Learning for Knowledge Graphs [96.73259297063619]
  We consider a novel formulation, zero-shot learning, to free this cumbersome curation. For newly-added relations, we attempt to learn their semantic features from their text descriptions. We leverage Generative Adrial Networks (GANs) to establish the connection between text and knowledge graph domain.
  arXiv  Detail & Related papers  (2020-01-08T01:19:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.