Optimizing Transformer based on high-performance optimizer for predicting employment sentiment in American social media content

• URL: http://arxiv.org/abs/2410.10874v1
• Date: Wed, 09 Oct 2024 03:14:05 GMT
• Title: Optimizing Transformer based on high-performance optimizer for predicting employment sentiment in American social media content
• Authors: Feiyang Wang, Qiaozhi Bao, Zixuan Wang, Yanlin Chen,
• Abstract summary: This article improves the Transformer model based on swarm intelligence optimization algorithm, aiming to predict the emotions of employment related text content on American social media. During the training process, the accuracy of the model gradually increased from 49.27% to 82.83%, while the loss value decreased from 0.67 to 0.35. The improved model proposed in this article not only improves the accuracy of sentiment recognition in employment related texts on social media, but also has important practical significance.
• Score: 9.49688045612671
• License:
• Abstract: This article improves the Transformer model based on swarm intelligence optimization algorithm, aiming to predict the emotions of employment related text content on American social media. Through text preprocessing, feature extraction, and vectorization, the text data was successfully converted into numerical data and imported into the model for training. The experimental results show that during the training process, the accuracy of the model gradually increased from 49.27% to 82.83%, while the loss value decreased from 0.67 to 0.35, indicating a significant improvement in the performance of the model on the training set. According to the confusion matrix analysis of the training set, the accuracy of the training set is 86.15%. The confusion matrix of the test set also showed good performance, with an accuracy of 82.91%. The accuracy difference between the training set and the test set is only 3.24%, indicating that the model has strong generalization ability. In addition, the evaluation of polygon results shows that the model performs well in classification accuracy, sensitivity, specificity, and area under the curve (AUC), with a Kappa coefficient of 0.66 and an F-measure of 0.80, further verifying the effectiveness of the model in social media sentiment analysis. The improved model proposed in this article not only improves the accuracy of sentiment recognition in employment related texts on social media, but also has important practical significance. This social media based data analysis method can not only capture social dynamics in a timely manner, but also promote decision-makers to pay attention to public concerns and provide data support for improving employment conditions.

Related papers

• Improving Embedding Accuracy for Document Retrieval Using Entity Relationship Maps and Model-Aware Contrastive Sampling [0.0]
  APEX-Embedding-7B is a 7-billion parameter decoder-only text Feature Extraction Model. Our approach employs two training techniques that yield an emergent improvement in factual focus. Based on our evaluations, our model establishes a new state-of-the-art standard in text feature extraction for longer context document retrieval tasks.
  arXiv  Detail & Related papers  (2024-10-08T17:36:48Z)
• Phrasing for UX: Enhancing Information Engagement through Computational Linguistics and Creative Analytics [0.0]
  This study explores the relationship between textual features and Information Engagement (IE) on digital platforms. It highlights the impact of computational linguistics and analytics on user interaction. The READ model is introduced to quantify key predictors like representativeness, ease of use, affect, and distribution.
  arXiv  Detail & Related papers  (2024-08-23T00:33:47Z)
• PUMA: margin-based data pruning [51.12154122266251]
  We focus on data pruning, where some training samples are removed based on the distance to the model classification boundary (i.e., margin) We propose PUMA, a new data pruning strategy that computes the margin using DeepFool. We show that PUMA can be used on top of the current state-of-the-art methodology in robustness, and it is able to significantly improve the model performance unlike the existing data pruning strategies.
  arXiv  Detail & Related papers  (2024-05-10T08:02:20Z)
• Text Quality-Based Pruning for Efficient Training of Language Models [66.66259229732121]
  We propose a novel method for numerically evaluating text quality in large unlabelled NLP datasets. By proposing the text quality metric, the paper establishes a framework to identify and eliminate low-quality text instances. Experimental results over multiple models and datasets demonstrate the efficacy of this approach.
  arXiv  Detail & Related papers  (2024-04-26T18:01:25Z)
• Text Sentiment Analysis and Classification Based on Bidirectional Gated Recurrent Units (GRUs) Model [6.096738978232722]
  This paper explores the importance of text sentiment analysis and classification in the field of natural language processing. It proposes a new approach to sentiment analysis and classification based on the bidirectional gated recurrent units (GRUs) model.
  arXiv  Detail & Related papers  (2024-04-26T02:40:03Z)
• Data Mixing Laws: Optimizing Data Mixtures by Predicting Language Modeling Performance [55.872926690722714]
  We study the predictability of model performance regarding the mixture proportions in function forms. We propose nested use of the scaling laws of training steps, model sizes, and our data mixing law. Our method effectively optimize the training mixture of a 1B model trained for 100B tokens in RedPajama.
  arXiv  Detail & Related papers  (2024-03-25T17:14:00Z)
• Robust Learning with Progressive Data Expansion Against Spurious Correlation [65.83104529677234]
  We study the learning process of a two-layer nonlinear convolutional neural network in the presence of spurious features. Our analysis suggests that imbalanced data groups and easily learnable spurious features can lead to the dominance of spurious features during the learning process. We propose a new training algorithm called PDE that efficiently enhances the model's robustness for a better worst-group performance.
  arXiv  Detail & Related papers  (2023-06-08T05:44:06Z)
• Boosting Visual-Language Models by Exploiting Hard Samples [126.35125029639168]
  HELIP is a cost-effective strategy tailored to enhance the performance of existing CLIP models. Our method allows for effortless integration with existing models' training pipelines. On comprehensive benchmarks, HELIP consistently boosts existing models to achieve leading performance.
  arXiv  Detail & Related papers  (2023-05-09T07:00:17Z)
• Presence of informal language, such as emoticons, hashtags, and slang, impact the performance of sentiment analysis models on social media text? [0.0]
  This study investigated the influence of informal language, such as emoticons and slang, on the performance of sentiment analysis models applied to social media text. A CNN model was developed and trained on three datasets: a sarcasm dataset, a sentiment dataset, and an emoticon dataset. The results revealed that the model achieved an accuracy of 96.47% on the sarcasm dataset, with the lowest accuracy for class 1. The amalgamation of sarcasm and sentiment datasets improved the accuracy of the model to 95.1%, and the addition of emoticon dataset has a slight positive impact on the accuracy of the model to 95.37%.
  arXiv  Detail & Related papers  (2023-01-28T22:21:51Z)
• Domain Adaptation of Transformer-Based Models using Unlabeled Data for Relevance and Polarity Classification of German Customer Feedback [1.2999413717930817]
  This work explores how efficient transformer-based models are when working with a German customer feedback dataset. The experimental results show that transformer-based models can reach significant improvements compared to a fastText baseline.
  arXiv  Detail & Related papers  (2022-12-12T08:32:28Z)
• To be Critical: Self-Calibrated Weakly Supervised Learning for Salient Object Detection [95.21700830273221]
  Weakly-supervised salient object detection (WSOD) aims to develop saliency models using image-level annotations. We propose a self-calibrated training strategy by explicitly establishing a mutual calibration loop between pseudo labels and network predictions. We prove that even a much smaller dataset with well-matched annotations can facilitate models to achieve better performance as well as generalizability.
  arXiv  Detail & Related papers  (2021-09-04T02:45:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.