VGCN-BERT: Augmenting BERT with Graph Embedding for Text Classification

• URL: http://arxiv.org/abs/2004.05707v1
• Date: Sun, 12 Apr 2020 22:02:33 GMT
• Title: VGCN-BERT: Augmenting BERT with Graph Embedding for Text Classification
• Authors: Zhibin Lu, Pan Du, Jian-Yun Nie
• Abstract summary: VGCN-BERT model combines the capability of BERT with a Vocabulary Graph Convolutional Network (VGCN) In our experiments on several text classification datasets, our approach outperforms BERT and GCN alone.
• Score: 21.96079052962283
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Much progress has been made recently on text classification with methods based on neural networks. In particular, models using attention mechanism such as BERT have shown to have the capability of capturing the contextual information within a sentence or document. However, their ability of capturing the global information about the vocabulary of a language is more limited. This latter is the strength of Graph Convolutional Networks (GCN). In this paper, we propose VGCN-BERT model which combines the capability of BERT with a Vocabulary Graph Convolutional Network (VGCN). Local information and global information interact through different layers of BERT, allowing them to influence mutually and to build together a final representation for classification. In our experiments on several text classification datasets, our approach outperforms BERT and GCN alone, and achieve higher effectiveness than that reported in previous studies.

Related papers

• Bridging Local Details and Global Context in Text-Attributed Graphs [62.522550655068336]
  GraphBridge is a framework that bridges local and global perspectives by leveraging contextual textual information. Our method achieves state-of-theart performance, while our graph-aware token reduction module significantly enhances efficiency and solves scalability issues.
  arXiv  Detail & Related papers  (2024-06-18T13:35:25Z)
• Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learning [51.90524745663737]
  A key innovation is our use of explanations as features, which can be used to boost GNN performance on downstream tasks. Our method achieves state-of-the-art results on well-established TAG datasets. Our method significantly speeds up training, achieving a 2.88 times improvement over the closest baseline on ogbn-arxiv.
  arXiv  Detail & Related papers  (2023-05-31T03:18:03Z)
• A semantic hierarchical graph neural network for text classification [1.439766998338892]
  We propose a new hierarchical graph neural network (HieGNN) which extracts corresponding information from word-level, sentence-level and document-level respectively. Experimental results on several benchmark datasets achieve better or similar results compared to several baseline methods.
  arXiv  Detail & Related papers  (2022-09-15T03:59:31Z)
• Understanding Graph Convolutional Networks for Text Classification [9.495731689143827]
  We conduct a comprehensive analysis of the role of node and edge embeddings in a graph and its GCN learning techniques in text classification. Our analysis is the first of its kind and provides useful insights into the importance of each graph node/edge construction mechanism.
  arXiv  Detail & Related papers  (2022-03-30T05:14:31Z)
• TextRGNN: Residual Graph Neural Networks for Text Classification [13.912147013558846]
  TextRGNN is an improved GNN structure that introduces residual connection to deepen the convolution network depth. Our structure can obtain a wider node receptive field and effectively suppress the over-smoothing of node features. It can significantly improve the classification accuracy whether in corpus level or text level, and achieve SOTA performance on a wide range of text classification datasets.
  arXiv  Detail & Related papers  (2021-12-30T13:48:58Z)
• Hierarchical Heterogeneous Graph Representation Learning for Short Text Classification [60.233529926965836]
  We propose a new method called SHINE, which is based on graph neural network (GNN) for short text classification. First, we model the short text dataset as a hierarchical heterogeneous graph consisting of word-level component graphs. Then, we dynamically learn a short document graph that facilitates effective label propagation among similar short texts.
  arXiv  Detail & Related papers  (2021-10-30T05:33:05Z)
• BERT4GCN: Using BERT Intermediate Layers to Augment GCN for Aspect-based Sentiment Classification [2.982218441172364]
  Graph-based Sentiment Classification (ABSC) approaches have yielded state-of-the-art results, expecially when equipped with contextual word embedding from pre-training language models (PLMs) We propose a novel model, BERT4GCN, which integrates the grammatical sequential features from the PLM of BERT, and the syntactic knowledge from dependency graphs.
  arXiv  Detail & Related papers  (2021-10-01T02:03:43Z)
• BertGCN: Transductive Text Classification by Combining GCN and BERT [33.866453485862124]
  BertGCN is a model that combines large scale pretraining and transductive learning for text classification. BertGCN SOTA achieves performances on a wide range of text classification datasets.
  arXiv  Detail & Related papers  (2021-05-12T15:20:01Z)
• GraphFormers: GNN-nested Transformers for Representation Learning on Textual Graph [53.70520466556453]
  We propose GraphFormers, where layerwise GNN components are nested alongside the transformer blocks of language models. With the proposed architecture, the text encoding and the graph aggregation are fused into an iterative workflow. In addition, a progressive learning strategy is introduced, where the model is successively trained on manipulated data and original data to reinforce its capability of integrating information on graph.
  arXiv  Detail & Related papers  (2021-05-06T12:20:41Z)
• Graph Convolutional Network for Swahili News Classification [78.6363825307044]
  This work empirically demonstrates the ability of Text Graph Convolutional Network (Text GCN) to outperform traditional natural language processing benchmarks for the task of semi-supervised Swahili news classification.
  arXiv  Detail & Related papers  (2021-03-16T21:03:47Z)
• Multi-Level Graph Convolutional Network with Automatic Graph Learning for Hyperspectral Image Classification [63.56018768401328]
  We propose a Multi-level Graph Convolutional Network (GCN) with Automatic Graph Learning method (MGCN-AGL) for HSI classification. By employing attention mechanism to characterize the importance among spatially neighboring regions, the most relevant information can be adaptively incorporated to make decisions. Our MGCN-AGL encodes the long range dependencies among image regions based on the expressive representations that have been produced at local level.
  arXiv  Detail & Related papers  (2020-09-19T09:26:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.