BERT-Based Combination of Convolutional and Recurrent Neural Network for Indonesian Sentiment Analysis

• URL: http://arxiv.org/abs/2211.05273v1
• Date: Thu, 10 Nov 2022 00:32:40 GMT
• Title: BERT-Based Combination of Convolutional and Recurrent Neural Network for Indonesian Sentiment Analysis
• Authors: Hendri Murfi, Syamsyuriani, Theresia Gowandi, Gianinna Ardaneswari, Siti Nurrohmah
• Abstract summary: This research extends the previous hybrid deep learning using BERT representation for Indonesian sentiment analysis. Our simulation shows that the BERT representation improves the accuracies of all hybrid architectures.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Sentiment analysis is the computational study of opinions and emotions ex-pressed in text. Deep learning is a model that is currently producing state-of-the-art in various application domains, including sentiment analysis. Many researchers are using a hybrid approach that combines different deep learning models and has been shown to improve model performance. In sentiment analysis, input in text data is first converted into a numerical representation. The standard method used to obtain a text representation is the fine-tuned embedding method. However, this method does not pay attention to each word's context in the sentence. Therefore, the Bidirectional Encoder Representation from Transformer (BERT) model is used to obtain text representations based on the context and position of words in sentences. This research extends the previous hybrid deep learning using BERT representation for Indonesian sentiment analysis. Our simulation shows that the BERT representation improves the accuracies of all hybrid architectures. The BERT-based LSTM-CNN also reaches slightly better accuracies than other BERT-based hybrid architectures.

Related papers

• Towards Fast Adaptation of Pretrained Contrastive Models for Multi-channel Video-Language Retrieval [70.30052749168013]
  Multi-channel video-language retrieval require models to understand information from different channels. contrastive multimodal models are shown to be highly effective at aligning entities in images/videos and text. There is not a clear way to quickly adapt these two lines to multi-channel video-language retrieval with limited data and resources.
  arXiv  Detail & Related papers  (2022-06-05T01:43:52Z)
• Hierarchical Neural Network Approaches for Long Document Classification [3.6700088931938835]
  We employ pre-trained Universal Sentence (USE) and Bidirectional Representations from Transformers (BERT) in a hierarchical setup to capture better representations efficiently. Our proposed models are conceptually simple where we divide the input data into chunks and then pass this through base models of BERT and USE. We show that USE + CNN/LSTM performs better than its stand-alone baseline. Whereas the BERT + CNN/LSTM performs on par with its stand-alone counterpart.
  arXiv  Detail & Related papers  (2022-01-18T07:17:40Z)
• HETFORMER: Heterogeneous Transformer with Sparse Attention for Long-Text Extractive Summarization [57.798070356553936]
  HETFORMER is a Transformer-based pre-trained model with multi-granularity sparse attentions for extractive summarization. Experiments on both single- and multi-document summarization tasks show that HETFORMER achieves state-of-the-art performance in Rouge F1.
  arXiv  Detail & Related papers  (2021-10-12T22:42:31Z)
• Pre-training Language Model Incorporating Domain-specific Heterogeneous Knowledge into A Unified Representation [49.89831914386982]
  We propose a unified pre-trained language model (PLM) for all forms of text, including unstructured text, semi-structured text, and well-structured text. Our approach outperforms the pre-training of plain text using only 1/4 of the data.
  arXiv  Detail & Related papers  (2021-09-02T16:05:24Z)
• BERT based sentiment analysis: A software engineering perspective [0.9176056742068814]
  The paper presents three different strategies to analyse BERT based model for sentiment analysis. The experimental results show that the BERT based ensemble approach and the compressed BERT model attain improvements by 6-12% over prevailing tools for the F1 measure on all three datasets.
  arXiv  Detail & Related papers  (2021-06-04T16:28:26Z)
• Dependency Parsing based Semantic Representation Learning with Graph Neural Network for Enhancing Expressiveness of Text-to-Speech [49.05471750563229]
  We propose a semantic representation learning method based on graph neural network, considering dependency relations of a sentence. We show that our proposed method outperforms the baseline using vanilla BERT features both in LJSpeech and Bilzzard Challenge 2013 datasets.
  arXiv  Detail & Related papers  (2021-04-14T13:09:51Z)
• Learning to Synthesize Data for Semantic Parsing [57.190817162674875]
  We propose a generative model which models the composition of programs and maps a program to an utterance. Due to the simplicity of PCFG and pre-trained BART, our generative model can be efficiently learned from existing data at hand. We evaluate our method in both in-domain and out-of-domain settings of text-to-Query parsing on the standard benchmarks of GeoQuery and Spider.
  arXiv  Detail & Related papers  (2021-04-12T21:24:02Z)
• Fine-Tuning BERT for Sentiment Analysis of Vietnamese Reviews [0.0]
  Experimental results on two datasets show thatmodels using BERT slightly outperform other models usingGloVe and FastText. Our proposed BERT fine-tuning method produces amodel with better performance than the original BERT fine-tuning method.
  arXiv  Detail & Related papers  (2020-11-20T14:45:46Z)
• Bidirectional Encoder Representations from Transformers (BERT): A sentiment analysis odyssey [0.0]
  The study puts forth two key insights: (1) relative efficacy of four highly advanced and widely used sentiment analysis techniques; and (2) undisputed superiority of pre-trained advanced supervised deep learning BERT model in sentiment analysis from text data. We use publicly available labeled corpora of 50,000 movie reviews originally posted on internet movie database (IMDB) for analysis using Sent WordNet lexicon, logistic regression, LSTM, and BERT.
  arXiv  Detail & Related papers  (2020-07-02T14:23:57Z)
• POINTER: Constrained Progressive Text Generation via Insertion-based Generative Pre-training [93.79766670391618]
  We present POINTER, a novel insertion-based approach for hard-constrained text generation. The proposed method operates by progressively inserting new tokens between existing tokens in a parallel manner. The resulting coarse-to-fine hierarchy makes the generation process intuitive and interpretable.
  arXiv  Detail & Related papers  (2020-05-01T18:11:54Z)
• Abstractive Text Summarization based on Language Model Conditioning and Locality Modeling [4.525267347429154]
  We train a Transformer-based neural model on the BERT language model. In addition, we propose a new method of BERT-windowing, which allows chunk-wise processing of texts longer than the BERT window size. The results of our models are compared to a baseline and the state-of-the-art models on the CNN/Daily Mail dataset.
  arXiv  Detail & Related papers  (2020-03-29T14:00:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.