Language Semantics Interpretation with an Interaction-based Recurrent Neural Networks

• URL: http://arxiv.org/abs/2112.02997v1
• Date: Tue, 2 Nov 2021 00:39:21 GMT
• Title: Language Semantics Interpretation with an Interaction-based Recurrent Neural Networks
• Authors: Shaw-Hwa Lo, Yiqiao Yin
• Abstract summary: This paper proposes a novel influence score (I-score), a greedy search algorithm called Backward Dropping Algorithm (BDA), and a novel feature engineering technique called the "dagger technique" The proposed methods are applied to improve prediction performance with an 81% error reduction comparing with other popular peers.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Text classification is a fundamental language task in Natural Language Processing. A variety of sequential models is capable making good predictions yet there is lack of connection between language semantics and prediction results. This paper proposes a novel influence score (I-score), a greedy search algorithm called Backward Dropping Algorithm (BDA), and a novel feature engineering technique called the "dagger technique". First, the paper proposes a novel influence score (I-score) to detect and search for the important language semantics in text document that are useful for making good prediction in text classification tasks. Next, a greedy search algorithm called the Backward Dropping Algorithm is proposed to handle long-term dependencies in the dataset. Moreover, the paper proposes a novel engineering technique called the "dagger technique" that fully preserve the relationship between explanatory variable and response variable. The proposed techniques can be further generalized into any feed-forward Artificial Neural Networks (ANNs) and Convolutional Neural Networks (CNNs), and any neural network. A real-world application on the Internet Movie Database (IMDB) is used and the proposed methods are applied to improve prediction performance with an 81% error reduction comparing with other popular peers if I-score and "dagger technique" are not implemented.

Related papers

• Graph Neural Network Approach to Semantic Type Detection in Tables [3.929053351442136]
  This study addresses the challenge of detecting semantic column types in relational tables. We propose a novel approach using Graph Neural Networks (GNNs) to model intra-table dependencies. Our proposed method not only outperforms existing state-of-the-art algorithms but also offers novel insights into the utility and functionality of various GNN types for semantic type detection.
  arXiv  Detail & Related papers  (2024-04-30T18:17:44Z)
• A Novel Ehanced Move Recognition Algorithm Based on Pre-trained Models with Positional Embeddings [6.688643243555054]
  The recognition of abstracts is crucial for effectively locating the content and clarifying the article. This paper proposes a novel enhanced move recognition algorithm with an improved pre-trained model and a gated network with attention mechanism for unstructured abstracts of Chinese scientific and technological papers.
  arXiv  Detail & Related papers  (2023-08-14T03:20:28Z)
• SpellMapper: A non-autoregressive neural spellchecker for ASR customization with candidate retrieval based on n-gram mappings [76.87664008338317]
  Contextual spelling correction models are an alternative to shallow fusion to improve automatic speech recognition. We propose a novel algorithm for candidate retrieval based on misspelled n-gram mappings. Experiments on Spoken Wikipedia show 21.4% word error rate improvement compared to a baseline ASR system.
  arXiv  Detail & Related papers  (2023-06-04T10:00:12Z)
• Towards Better Out-of-Distribution Generalization of Neural Algorithmic Reasoning Tasks [51.8723187709964]
  We study the OOD generalization of neural algorithmic reasoning tasks. The goal is to learn an algorithm from input-output pairs using deep neural networks.
  arXiv  Detail & Related papers  (2022-11-01T18:33:20Z)
• A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyond [145.43029264191543]
  Non-autoregressive (NAR) generation is first proposed in machine translation (NMT) to speed up inference. While NAR generation can significantly accelerate machine translation, the inference of autoregressive (AR) generation sacrificed translation accuracy. Many new models and algorithms have been designed/proposed to bridge the accuracy gap between NAR generation and AR generation.
  arXiv  Detail & Related papers  (2022-04-20T07:25:22Z)
• Semantic-Preserving Adversarial Text Attacks [85.32186121859321]
  We propose a Bigram and Unigram based adaptive Semantic Preservation Optimization (BU-SPO) method to examine the vulnerability of deep models. Our method achieves the highest attack success rates and semantics rates by changing the smallest number of words compared with existing methods.
  arXiv  Detail & Related papers  (2021-08-23T09:05:18Z)
• Accelerating Text Mining Using Domain-Specific Stop Word Lists [57.76576681191192]
  We present a novel approach for the automatic extraction of domain-specific words called the hyperplane-based approach. The hyperplane-based approach can significantly reduce text dimensionality by eliminating irrelevant features. Results indicate that the hyperplane-based approach can reduce the dimensionality of the corpus by 90% and outperforms mutual information.
  arXiv  Detail & Related papers  (2020-11-18T17:42:32Z)
• Be More with Less: Hypergraph Attention Networks for Inductive Text Classification [56.98218530073927]
  Graph neural networks (GNNs) have received increasing attention in the research community and demonstrated their promising results on this canonical task. Despite the success, their performance could be largely jeopardized in practice since they are unable to capture high-order interaction between words. We propose a principled model -- hypergraph attention networks (HyperGAT) which can obtain more expressive power with less computational consumption for text representation learning.
  arXiv  Detail & Related papers  (2020-11-01T00:21:59Z)
• SHAP values for Explaining CNN-based Text Classification Models [10.881494765759829]
  This paper develops a methodology to compute SHAP values for local explainability of CNN-based text classification models. The approach is also extended to compute global scores to assess the importance of features.
  arXiv  Detail & Related papers  (2020-08-26T21:28:41Z)
• SeMemNN: A Semantic Matrix-Based Memory Neural Network for Text Classification [15.111940377403252]
  We propose 5 different configurations for the semantic matrix-based memory neural network with end-to-end learning manner. We evaluate our proposed method on two corpora of news articles (AG news, Sogou news)
  arXiv  Detail & Related papers  (2020-03-04T02:00:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.