Self-attention-based BiGRU and capsule network for named entity recognition

• URL: http://arxiv.org/abs/2002.00735v1
• Date: Thu, 30 Jan 2020 21:51:58 GMT
• Title: Self-attention-based BiGRU and capsule network for named entity recognition
• Authors: Jianfeng Deng and Lianglun Cheng and Zhuowei Wang
• Abstract summary: We propose a self-attention-based bidirectional gated recurrent unit(BiGRU) and capsule network(CapsNet) for NER. BiGRU is used to capture sequence context features, and self-attention mechanism is proposed to give different focus on the information captured by hidden layer of BiGRU. We evaluate the recognition performance of the model on two datasets.
• Score: 1.8348489257164355
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Named entity recognition(NER) is one of the tasks of natural language processing(NLP). In view of the problem that the traditional character representation ability is weak and the neural network method is unable to capture the important sequence information. An self-attention-based bidirectional gated recurrent unit(BiGRU) and capsule network(CapsNet) for NER is proposed. This model generates character vectors through bidirectional encoder representation of transformers(BERT) pre-trained model. BiGRU is used to capture sequence context features, and self-attention mechanism is proposed to give different focus on the information captured by hidden layer of BiGRU. Finally, we propose to use CapsNet for entity recognition. We evaluated the recognition performance of the model on two datasets. Experimental results show that the model has better performance without relying on external dictionary information.

Related papers

• Knowledge-Informed Neural Network for Complex-Valued SAR Image Recognition [51.03674130115878]
  We introduce the Knowledge-Informed Neural Network (KINN), a lightweight framework built upon a novel "compression-aggregation-compression" architecture.<n>KINN establishes a state-of-the-art in parameter-efficient recognition, offering exceptional generalization in data-scarce and out-of-distribution scenarios.
  arXiv  Detail & Related papers  (2025-10-23T07:12:26Z)
• Information-Bottleneck Driven Binary Neural Network for Change Detection [53.866667209237434]
  Binarized Change Detection (BiCD) is the first binary neural network (BNN) designed specifically for change detection.<n>We introduce an auxiliary objective based on the Information Bottleneck (IB) principle, guiding the encoder to retain essential input information.<n>BiCD establishes a new benchmark for BNN-based change detection, achieving state-of-the-art performance in this domain.
  arXiv  Detail & Related papers  (2025-07-04T11:56:16Z)
• CBGT-Net: A Neuromimetic Architecture for Robust Classification of Streaming Data [0.994853090657971]
  The CBGT-Net learns to produce an output after a sufficient criteria for evidence is achieved from a stream of observed data. We show that the CBGT-Net provides improved accuracy and robustness compared to models trained to classify from a single patch.
  arXiv  Detail & Related papers  (2024-03-24T00:46:40Z)
• With a Little Help from your own Past: Prototypical Memory Networks for Image Captioning [47.96387857237473]
  We devise a network which can perform attention over activations obtained while processing other training samples. Our memory models the distribution of past keys and values through the definition of prototype vectors. We demonstrate that our proposal can increase the performance of an encoder-decoder Transformer by 3.7 CIDEr points both when training in cross-entropy only and when fine-tuning with self-critical sequence training.
  arXiv  Detail & Related papers  (2023-08-23T18:53:00Z)
• Brain Network Transformer [13.239896897835191]
  We study Transformer-based models for brain network analysis. Driven by the unique properties of data, we model brain networks as graphs with nodes of fixed size and order. We re-standardize the evaluation pipeline on the only one publicly available large-scale brain network dataset of ABIDE.
  arXiv  Detail & Related papers  (2022-10-13T02:30:06Z)
• Dynamic Prototype Mask for Occluded Person Re-Identification [88.7782299372656]
  Existing methods mainly address this issue by employing body clues provided by an extra network to distinguish the visible part. We propose a novel Dynamic Prototype Mask (DPM) based on two self-evident prior knowledge. Under this condition, the occluded representation could be well aligned in a selected subspace spontaneously.
  arXiv  Detail & Related papers  (2022-07-19T03:31:13Z)
• Research on Dual Channel News Headline Classification Based on ERNIE Pre-training Model [13.222137788045416]
  The proposed model improves the accuracy, precision and F1-score of news headline classification compared with the traditional neural network model. It can perform well in the multi-classification application of news headline text under large data volume.
  arXiv  Detail & Related papers  (2022-02-14T10:44:12Z)
• Auto-Parsing Network for Image Captioning and Visual Question Answering [101.77688388554097]
  We propose an Auto-Parsing Network (APN) to discover and exploit the input data's hidden tree structures. Specifically, we impose a Probabilistic Graphical Model (PGM) parameterized by the attention operations on each self-attention layer to incorporate sparse assumption.
  arXiv  Detail & Related papers  (2021-08-24T08:14:35Z)
• A Driving Behavior Recognition Model with Bi-LSTM and Multi-Scale CNN [59.57221522897815]
  We propose a neural network model based on trajectories information for driving behavior recognition. We evaluate the proposed model on the public BLVD dataset, achieving a satisfying performance.
  arXiv  Detail & Related papers  (2021-03-01T06:47:29Z)
• Scribble-Supervised Semantic Segmentation by Random Walk on Neural Representation and Self-Supervision on Neural Eigenspace [10.603823180750446]
  This work aims to achieve semantic segmentation supervised by scribble label directly without auxiliary information and other intermediate manipulation. We impose diffusion on neural representation by random walk and consistency on neural eigenspace by self-supervision. The results demonstrate the superiority of the proposed method and are even comparable to some full-label supervised ones.
  arXiv  Detail & Related papers  (2020-11-11T08:22:25Z)
• BiDet: An Efficient Binarized Object Detector [96.19708396510894]
  We propose a binarized neural network learning method called BiDet for efficient object detection. Our BiDet fully utilizes the representational capacity of the binary neural networks for object detection by redundancy removal. Our method outperforms the state-of-the-art binary neural networks by a sizable margin.
  arXiv  Detail & Related papers  (2020-03-09T08:16:16Z)
• Learn to Predict Sets Using Feed-Forward Neural Networks [63.91494644881925]
  This paper addresses the task of set prediction using deep feed-forward neural networks. We present a novel approach for learning to predict sets with unknown permutation and cardinality. We demonstrate the validity of our set formulations on relevant vision problems.
  arXiv  Detail & Related papers  (2020-01-30T01:52:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.