Large Language Models in Targeted Sentiment Analysis

• URL: http://arxiv.org/abs/2404.12342v1
• Date: Thu, 18 Apr 2024 17:16:16 GMT
• Title: Large Language Models in Targeted Sentiment Analysis
• Abstract summary: We study sentiment analysis capabilities of instruction-tuned large language models (LLMs) The best results of sentiment analysis on RuSentNE-2023 were achieved by fine-tuned Flan-T5-xl.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper we investigate the use of decoder-based generative transformers for extracting sentiment towards the named entities in Russian news articles. We study sentiment analysis capabilities of instruction-tuned large language models (LLMs). We consider the dataset of RuSentNE-2023 in our study. The first group of experiments was aimed at the evaluation of zero-shot capabilities of LLMs with closed and open transparencies. The second covers the fine-tuning of Flan-T5 using the "chain-of-thought" (CoT) three-hop reasoning framework (THoR). We found that the results of the zero-shot approaches are similar to the results achieved by baseline fine-tuned encoder-based transformers (BERT-base). Reasoning capabilities of the fine-tuned Flan-T5 models with THoR achieve at least 5% increment with the base-size model compared to the results of the zero-shot experiment. The best results of sentiment analysis on RuSentNE-2023 were achieved by fine-tuned Flan-T5-xl, which surpassed the results of previous state-of-the-art transformer-based classifiers. Our CoT application framework is publicly available: https://github.com/nicolay-r/Reasoning-for-Sentiment-Analysis-Framework

Related papers

• Architecture-Dependent Causal Transfer of Activation States Across Large Language Models [51.56484100374058]
  Internal language activation states can be transferred causally between different large language models.<n>Injecting projected activations into a target model during generation produces a statistically significant, preregistered causal effect.<n>We conclude that end-to-end activation-state transfer between LLMs, as currently implemented, is architecture-dependent rather than universal.
  arXiv  Detail & Related papers  (2026-08-17T09:53:27Z)
• Context-Instrumental Data Distillation for Kubernetes Manifest Generation: Method and Experimental Evaluation [36.94429692322632]
  This paper examines the specialization of Small Language Models with up to 4 billion parameters for generating artifacts in domain-specific languages (AML)<n>We propose the context-instrumental data distillation method: the source corpus is formed through synthetic generation and, in an extended scheme, through reverse instruction generation from real Y files.
  arXiv  Detail & Related papers  (2026-05-25T13:30:38Z)
• Benchmarking Logistic Regression, SVM, Naive Bayes, and IndoBERT Fine-Tuning for Sentiment Analysis on Indonesian Product Reviews [0.0]
  This paper benchmarks traditional Machine Learning approaches against a Transformer-based Deep Learning model for a three-class sentiment analysis task.<n>We fine-tuned the IndoBERT model for contextual sequence classification.<n>We demonstrate the practical viability of our pipeline by deploying the final sentiment classification model as an interactive Gradio web application.
  arXiv  Detail & Related papers  (2026-05-05T07:20:42Z)
• HausaMovieReview: A Benchmark Dataset for Sentiment Analysis in Low-Resource African Language [1.3465808629549525]
  This paper introduces a novel benchmark dataset comprising 5,000 YouTube comments in Hausa and code-switched English.<n>We use this dataset to conduct a comparative analysis of classical models and fine-tuned transformer models.<n>Our results reveal a key finding: the Decision Tree classifier, with an accuracy and F1-score 89.72% and 89.60% respectively, significantly outperformed the deep learning models.
  arXiv  Detail & Related papers  (2025-09-17T22:57:21Z)
• Transformer Enhanced Relation Classification: A Comparative Analysis of Contextuality, Data Efficiency and Sequence Complexity [9.402566546100973]
  We systematically compare the performance of deep supervised learning approaches without transformers and those with transformers.<n>The results show that transformer-based models outperform non-transformer models, achieving micro F1 scores of 80-90%.<n>We briefly review the research journey in supervised relation classification and discuss the role and current status of large language models (LLMs) in relation extraction.
  arXiv  Detail & Related papers  (2025-09-14T18:11:31Z)
• Teaching LLM to Reason: Reinforcement Learning from Algorithmic Problems without Code [76.80306464249217]
  We propose TeaR, which aims at teaching LLMs to reason better.<n>TeaR leverages careful data curation and reinforcement learning to guide models in discovering optimal reasoning paths through code-related tasks.<n>We conduct extensive experiments using two base models and three long-CoT distillation models, with model sizes ranging from 1.5 billion to 32 billion parameters, and across 17 benchmarks spanning Math, Knowledge, Code, and Logical Reasoning.
  arXiv  Detail & Related papers  (2025-07-10T07:34:05Z)
• Sentiment Analysis in Software Engineering: Evaluating Generative Pre-trained Transformers [0.0]
  This study systematically evaluates the performance of bidirectional transformers, such as BERT, against generative pre-trained transformers, specifically GPT-4o-mini, in SE sentiment analysis.<n>The results reveal that fine-tuned GPT-4o-mini performs comparable to BERT and other bidirectional models on structured and balanced datasets like GitHub and Jira.<n>On linguistically complex datasets with imbalanced sentiment distributions, such as Stack Overflow, the default GPT-4o-mini model exhibits superior generalization, achieving an accuracy of 85.3% compared to the fine-tuned model's 13.1%.
  arXiv  Detail & Related papers  (2025-04-22T14:19:25Z)
• Reasoning-Based AI for Startup Evaluation (R.A.I.S.E.): A Memory-Augmented, Multi-Step Decision Framework [0.0]
  We present a novel framework that bridges the gap between the interpretability of decision trees and the advanced reasoning capabilities of large language models (LLMs) to predict startup success. Our approach leverages chain-of-thought prompting to generate detailed reasoning logs, which are subsequently distilled into structured, human-understandable logical rules. Our method not only augments traditional decision-making processes but also facilitates expert intervention and continuous policy refinement.
  arXiv  Detail & Related papers  (2025-04-16T13:53:42Z)
• FlanEC: Exploring Flan-T5 for Post-ASR Error Correction [25.931773686829796]
  We present an encoder-decoder model leveraging Flan-T5 for post-Automatic Speech Recognition (ASR) Generative Speech Error Correction (GenSEC) We explore its application within the GenSEC framework to enhance ASR outputs by mapping n-best hypotheses into a single output sentence. Specifically, we investigate whether scaling the training data and incorporating diverse datasets can lead to significant improvements in post-ASR error correction.
  arXiv  Detail & Related papers  (2025-01-22T16:06:04Z)
• Quantized Transformer Language Model Implementations on Edge Devices [1.2979415757860164]
  Large-scale transformer-based models like the Bidirectional Representations from Transformers (BERT) are widely used for Natural Language Processing (NLP) applications. These models are initially pre-trained with a large corpus with millions of parameters and then fine-tuned for a downstream NLP task. One of the major limitations of these large-scale models is that they cannot be deployed on resource-constrained devices due to their large model size and increased inference latency.
  arXiv  Detail & Related papers  (2023-10-06T01:59:19Z)
• Trainable Transformer in Transformer [48.754918968374334]
  We propose an efficient construction, Transformer in Transformer (in short, TinT), that allows a transformer to simulate and fine-tune complex models internally during inference. TinT accommodates many common transformer variants and its design ideas also improve the efficiency of past instantiations of simple models inside transformers. These findings suggest that large pre-trained language models are capable of performing intricate inferences.
  arXiv  Detail & Related papers  (2023-07-03T17:53:39Z)
• Model-Generated Pretraining Signals Improves Zero-Shot Generalization of Text-to-Text Transformers [98.30298332661323]
  This paper explores the effectiveness of model-generated signals in improving zero-shot generalization of text-to-text Transformers such as T5. We develop a new model, METRO-T0, which is pretrained using the redesigned ELECTRA-Style pretraining strategies and then prompt-finetuned on a mixture of NLP tasks. Our analysis on model's neural activation and parameter sensitivity reveals that the effectiveness of METRO-T0 stems from more balanced contribution of parameters and better utilization of their capacity.
  arXiv  Detail & Related papers  (2023-05-21T21:06:23Z)
• Transformer-based approaches to Sentiment Detection [55.41644538483948]
  We examined the performance of four different types of state-of-the-art transformer models for text classification. The RoBERTa transformer model performs best on the test dataset with a score of 82.6% and is highly recommended for quality predictions.
  arXiv  Detail & Related papers  (2023-03-13T17:12:03Z)
• The Flan Collection: Designing Data and Methods for Effective Instruction Tuning [118.70716915295091]
  We study the design decisions of publicly available instruction tuning methods, and break down the development of Flan 2022. We find task balancing and enrichment techniques are overlooked but critical to effective instruction tuning. To accelerate research on instruction tuning, we make the Flan 2022 collection of datasets, templates, and methods publicly available.
  arXiv  Detail & Related papers  (2023-01-31T15:03:44Z)
• T5Score: Discriminative Fine-tuning of Generative Evaluation Metrics [94.69907794006826]
  We present a framework that combines the best of both worlds, using both supervised and unsupervised signals from whatever data we have available. We operationalize this idea by training T5Score, a metric that uses these training signals with mT5 as the backbone. T5Score achieves the best performance on all datasets against existing top-scoring metrics at the segment level.
  arXiv  Detail & Related papers  (2022-12-12T06:29:04Z)
• Unifying Language Learning Paradigms [96.35981503087567]
  We present a unified framework for pre-training models that are universally effective across datasets and setups. We show how different pre-training objectives can be cast as one another and how interpolating between different objectives can be effective. Our model also achieve strong results at in-context learning, outperforming 175B GPT-3 on zero-shot SuperGLUE and tripling the performance of T5-XXL on one-shot summarization.
  arXiv  Detail & Related papers  (2022-05-10T19:32:20Z)
• Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers [57.931830650323]
  This paper presents scaling insights from pretraining and finetuning Transformers. We show that aside from only the model size, model shape matters for downstream fine-tuning. We present improved scaling protocols whereby our redesigned models achieve similar downstream fine-tuning quality.
  arXiv  Detail & Related papers  (2021-09-22T12:29:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.