Persian Slang Text Conversion to Formal and Deep Learning of Persian Short Texts on Social Media for Sentiment Classification

• URL: http://arxiv.org/abs/2403.06023v2
• Date: Tue, 3 Sep 2024 20:09:57 GMT
• Title: Persian Slang Text Conversion to Formal and Deep Learning of Persian Short Texts on Social Media for Sentiment Classification
• Authors: Mohsen Khazeni, Mohammad Heydari, Amir Albadvi,
• Abstract summary: PSC, Persian Slang Converter, is a tool for converting conversational texts into formal ones. Ten million unlabeled texts from various social networks and movie subtitles (as Conversational texts) have been used for training unsupervised models. 60,000 texts from the comments of Instagram social network users with positive, negative, and neutral labels are considered supervised data for training the emotion classification model.
• Score: 0.7373617024876725
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The lack of a suitable tool for the analysis of conversational texts in the Persian language has made various analyses of these texts, including Sentiment Analysis, difficult. In this research, we tried to make the understanding of these texts easier for the machine by providing PSC, Persian Slang Converter, a tool for converting conversational texts into formal ones, and by using the most up-to-date and best deep learning methods along with the PSC, the sentiment learning of short Persian language texts for the machine in a better way. be made More than 10 million unlabeled texts from various social networks and movie subtitles (as Conversational texts) and about 10 million news texts (as formal texts) have been used for training unsupervised models and formal implementation of the tool. 60,000 texts from the comments of Instagram social network users with positive, negative, and neutral labels are considered supervised data for training the emotion classification model of short texts. Using the formal tool, 57% of the words of the corpus of conversation were converted. Finally, by using the formalizer, FastText model, and deep LSTM network, an accuracy of 81.91 was obtained on the test data.

Related papers

• FarSSiBERT: A Novel Transformer-based Model for Semantic Similarity Measurement of Persian Social Networks Informal Texts [0.0]
  This paper introduces a new transformer-based model to measure semantic similarity between Persian informal short texts from social networks. It is pre-trained on approximately 104 million Persian informal short texts from social networks, making it one of a kind in the Persian language. It has been demonstrated that our proposed model outperforms ParsBERT, laBSE, and multilingual BERT in the Pearson and Spearman's coefficient criteria.
  arXiv  Detail & Related papers  (2024-07-27T05:04:49Z)
• MultiSocial: Multilingual Benchmark of Machine-Generated Text Detection of Social-Media Texts [0.6053347262128919]
  MultiSocial dataset contains 472,097 texts, of which about 58k are human-written. We use this benchmark to compare existing detection methods in zero-shot as well as fine-tuned form. Our results indicate that the fine-tuned detectors have no problem to be trained on social-media texts.
  arXiv  Detail & Related papers  (2024-06-18T12:26:09Z)
• Text Grouping Adapter: Adapting Pre-trained Text Detector for Layout Analysis [52.34110239735265]
  We present Text Grouping Adapter (TGA), a module that can enable the utilization of various pre-trained text detectors to learn layout analysis. Our comprehensive experiments demonstrate that, even with frozen pre-trained models, incorporating our TGA into various pre-trained text detectors and text spotters can achieve superior layout analysis performance.
  arXiv  Detail & Related papers  (2024-05-13T05:48:35Z)
• Emotion Classification in Short English Texts using Deep Learning Techniques [0.0]
  This study conducts a thorough examination of deep learning techniques for discerning emotions in short English texts. Deep learning approaches employ transfer learning and word embedding, notably BERT, to attain superior accuracy. "SmallEnglishEmotions" dataset comprises 6372 varied short English texts annotated with five primary emotion categories.
  arXiv  Detail & Related papers  (2024-02-25T09:17:22Z)
• Copy Is All You Need [66.00852205068327]
  We formulate text generation as progressively copying text segments from an existing text collection. Our approach achieves better generation quality according to both automatic and human evaluations. Our approach attains additional performance gains by simply scaling up to larger text collections.
  arXiv  Detail & Related papers  (2023-07-13T05:03:26Z)
• Distinguishing Human Generated Text From ChatGPT Generated Text Using Machine Learning [0.251657752676152]
  This paper presents a machine learning-based solution that can identify the ChatGPT delivered text from the human written text. We have tested the proposed model on a Kaggle dataset consisting of 10,000 texts out of which 5,204 texts were written by humans and collected from news and social media. On the corpus generated by GPT-3.5, the proposed algorithm presents an accuracy of 77%.
  arXiv  Detail & Related papers  (2023-05-26T09:27:43Z)
• Code-Switching Text Generation and Injection in Mandarin-English ASR [57.57570417273262]
  We investigate text generation and injection for improving the performance of an industry commonly-used streaming model, Transformer-Transducer (T-T) We first propose a strategy to generate code-switching text data and then investigate injecting generated text into T-T model explicitly by Text-To-Speech (TTS) conversion or implicitly by tying speech and text latent spaces. Experimental results on the T-T model trained with a dataset containing 1,800 hours of real Mandarin-English code-switched speech show that our approaches to inject generated code-switching text significantly boost the performance of T-T models.
  arXiv  Detail & Related papers  (2023-03-20T09:13:27Z)
• Language Matters: A Weakly Supervised Pre-training Approach for Scene Text Detection and Spotting [69.77701325270047]
  This paper presents a weakly supervised pre-training method that can acquire effective scene text representations. Our network consists of an image encoder and a character-aware text encoder that extract visual and textual features. Experiments show that our pre-trained model improves F-score by +2.5% and +4.8% while transferring its weights to other text detection and spotting networks.
  arXiv  Detail & Related papers  (2022-03-08T08:10:45Z)
• Leveraging Pre-trained Language Model for Speech Sentiment Analysis [58.78839114092951]
  We explore the use of pre-trained language models to learn sentiment information of written texts for speech sentiment analysis. We propose a pseudo label-based semi-supervised training strategy using a language model on an end-to-end speech sentiment approach.
  arXiv  Detail & Related papers  (2021-06-11T20:15:21Z)
• The Challenges of Persian User-generated Textual Content: A Machine Learning-Based Approach [0.0]
  This research applies machine learning-based approaches to tackle the hurdles that come with Persian user-generated textual content. The presented approach uses a machine-translated datasets to conduct sentiment analysis for the Persian language. The results of the experiments have shown promising state-of-the-art performance in contrast to the previous efforts.
  arXiv  Detail & Related papers  (2021-01-20T11:57:59Z)
• Text Perceptron: Towards End-to-End Arbitrary-Shaped Text Spotting [49.768327669098674]
  We propose an end-to-end trainable text spotting approach named Text Perceptron. It first employs an efficient segmentation-based text detector that learns the latent text reading order and boundary information. Then a novel Shape Transform Module (abbr. STM) is designed to transform the detected feature regions into regular morphologies.
  arXiv  Detail & Related papers  (2020-02-17T08:07:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.