Improve Document Embedding for Text Categorization Through Deep Siamese Neural Network

• URL: http://arxiv.org/abs/2006.00572v1
• Date: Sun, 31 May 2020 17:51:08 GMT
• Title: Improve Document Embedding for Text Categorization Through Deep Siamese Neural Network
• Authors: Erfaneh Gharavi, Hadi Veisi
• Abstract summary: Low-dimensional representation for text is one of main challenges for efficient natural language processing tasks. We propose the utilization of deep Siamese neural networks to map the documents with similar topics to a similar space in vector space representation. We show that the proposed representations outperform the conventional and state-of-the-art representations in the text classification task on this dataset.
• Score: 2.398608007786179
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Due to the increasing amount of data on the internet, finding a highly-informative, low-dimensional representation for text is one of the main challenges for efficient natural language processing tasks including text classification. This representation should capture the semantic information of the text while retaining their relevance level for document classification. This approach maps the documents with similar topics to a similar space in vector space representation. To obtain representation for large text, we propose the utilization of deep Siamese neural networks. To embed document relevance in topics in the distributed representation, we use a Siamese neural network to jointly learn document representations. Our Siamese network consists of two sub-network of multi-layer perceptron. We examine our representation for the text categorization task on BBC news dataset. The results show that the proposed representations outperform the conventional and state-of-the-art representations in the text classification task on this dataset.

Related papers

• Universal Multimodal Representation for Language Understanding [110.98786673598015]
  This work presents new methods to employ visual information as assistant signals to general NLP tasks. For each sentence, we first retrieve a flexible number of images either from a light topic-image lookup table extracted over the existing sentence-image pairs. Then, the text and images are encoded by a Transformer encoder and convolutional neural network, respectively.
  arXiv  Detail & Related papers  (2023-01-09T13:54:11Z)
• Attentive Deep Neural Networks for Legal Document Retrieval [2.4350217735794337]
  We study the use of attentive neural network-based text representation for statute law document retrieval. We develop two hierarchical architectures with sparse attention to represent long sentences and articles, and we name them Attentive CNN and Paraformer. Experimental results show that Attentive neural methods substantially outperform non-neural methods in terms of retrieval performance across datasets and languages.
  arXiv  Detail & Related papers  (2022-12-13T01:37:27Z)
• TeKo: Text-Rich Graph Neural Networks with External Knowledge [75.91477450060808]
  We propose a novel text-rich graph neural network with external knowledge (TeKo) We first present a flexible heterogeneous semantic network that incorporates high-quality entities. We then introduce two types of external knowledge, that is, structured triplets and unstructured entity description.
  arXiv  Detail & Related papers  (2022-06-15T02:33:10Z)
• Minimally-Supervised Structure-Rich Text Categorization via Learning on Text-Rich Networks [61.23408995934415]
  We propose a novel framework for minimally supervised categorization by learning from the text-rich network. Specifically, we jointly train two modules with different inductive biases -- a text analysis module for text understanding and a network learning module for class-discriminative, scalable network learning. Our experiments show that given only three seed documents per category, our framework can achieve an accuracy of about 92%.
  arXiv  Detail & Related papers  (2021-02-23T04:14:34Z)
• Deep Learning for Text Style Transfer: A Survey [71.8870854396927]
  Text style transfer is an important task in natural language generation, which aims to control certain attributes in the generated text. We present a systematic survey of the research on neural text style transfer, spanning over 100 representative articles since the first neural text style transfer work in 2017. We discuss the task formulation, existing datasets and subtasks, evaluation, as well as the rich methodologies in the presence of parallel and non-parallel data.
  arXiv  Detail & Related papers  (2020-11-01T04:04:43Z)
• Be More with Less: Hypergraph Attention Networks for Inductive Text Classification [56.98218530073927]
  Graph neural networks (GNNs) have received increasing attention in the research community and demonstrated their promising results on this canonical task. Despite the success, their performance could be largely jeopardized in practice since they are unable to capture high-order interaction between words. We propose a principled model -- hypergraph attention networks (HyperGAT) which can obtain more expressive power with less computational consumption for text representation learning.
  arXiv  Detail & Related papers  (2020-11-01T00:21:59Z)
• Improving Image Captioning with Better Use of Captions [65.39641077768488]
  We present a novel image captioning architecture to better explore semantics available in captions and leverage that to enhance both image representation and caption generation. Our models first construct caption-guided visual relationship graphs that introduce beneficial inductive bias using weakly supervised multi-instance learning. During generation, the model further incorporates visual relationships using multi-task learning for jointly predicting word and object/predicate tag sequences.
  arXiv  Detail & Related papers  (2020-06-21T14:10:47Z)
• From text saliency to linguistic objects: learning linguistic interpretable markers with a multi-channels convolutional architecture [2.064612766965483]
  We propose a novel approach to inspect the hidden layers of a fitted CNN in order to extract interpretable linguistic objects from texts exploiting classification process. We empirically demonstrate the efficiency of our approach on corpora from two different languages: English and French.
  arXiv  Detail & Related papers  (2020-04-07T10:46:58Z)
• Inductive Document Network Embedding with Topic-Word Attention [5.8010446129208155]
  Document network embedding aims at learning representations for a structured text corpus when documents are linked to each other. Recent algorithms extend network embedding approaches by incorporating the text content associated with the nodes in their formulations. In this paper, we propose an interpretable and inductive document network embedding method.
  arXiv  Detail & Related papers  (2020-01-10T10:14:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.