Fine-tuning with Hierarchical Prompting for Robust Propaganda Classification Across Annotation Schemas

• URL: http://arxiv.org/abs/2605.13663v1
• Date: Wed, 13 May 2026 15:19:46 GMT
• Title: Fine-tuning with Hierarchical Prompting for Robust Propaganda Classification Across Annotation Schemas
• Abstract summary: Propaganda detection in social media is challenging due to noisy, short texts and low annotation agreements.<n>We introduce a new intent-focused taxonomy of propaganda techniques and compare it against an established, higher-agreement schema.<n>Our results show that fine-tuning is essential, since it transforms weak zero-shot baselines into competitive systems.
• Score: 7.380400315626569
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Propaganda detection in social media is challenging due to noisy, short texts and low annotation agreements. We introduce a new intent-focused taxonomy of propaganda techniques and compare it against an established, higher-agreement schema. Along three dimensions (model portfolio, schema effects, and prompting strategy) we evaluate the taxonomies as a classification task with the help of four language models (GPT-4.1-nano, Phi-4 14B, Qwen2.5-14B, Qwen3-14B). Our results show that fine-tuning is essential, since it transforms weak zero-shot baselines into competitive systems and reveals methodological differences that are hidden using base models. Across schemas, the Qwen models achieve the strongest overall performance, and Phi-4 14B consistently outperforms GPT-4.1-nano. Our hierarchical prompting method (HiPP), which predicts fine-grained techniques before aggregating them, is especially beneficial after fine-tuning and on the more ambiguous, low-agreement taxonomy, while remaining competitive on the simpler schema. The HQP dataset, annotated with the new intent-based labels, provides a richer lens on propaganda's strategic goals and a challenging benchmark for future work on robust, real-world detection.

Related papers

• GIANTS: Generative Insight Anticipation from Scientific Literature [84.95947892931142]
  We introduce insight anticipation, a generation task in which a model predicts a downstream paper's core insight from its foundational parent papers.<n>We evaluate models using an LM judge that scores similarity between generated and ground-truth insights, and show that these similarity scores correlate with expert human ratings.<n>We present GIANTS-4B, an LM trained via reinforcement learning (RL) to optimize insight anticipation using these similarity scores as a proxy reward.
  arXiv  Detail & Related papers  (2026-04-10T18:13:55Z)
• Evaluating Prompting Strategies for Chart Question Answering with Large Language Models [2.243206998586006]
  We present a systematic evaluation of four widely used prompting paradigms (Zero-Shot, Few-Shot, Zero-Shot Chain-of-Thought, and Few-Shot Chain-of-Thought) across GPT-3.5, GPT-4, and GPT-4o.<n>Our framework operates exclusively on structured chart data, isolating prompt structure as the only experimental variable, and evaluates performance using two metrics: Accuracy and Exact Match.
  arXiv  Detail & Related papers  (2026-03-03T03:28:11Z)
• UM_FHS at TREC 2024 PLABA: Exploration of Fine-tuning and AI agent approach for plain language adaptations of biomedical text [3.223303935767146]
  This paper describes our submissions to the TREC 2024 PLABA track with the aim to simplify biomedical abstracts for a K8-level audience (13-14 years old students).<n>We tested three approaches using OpenAI's GPt-4o and GPt-4o-mini models: baseline prompt engineering, a two-AI agent approach, and fine-tuning.<n>Results indicated that the two-agent approach and baseline prompt engineering with GPt-4o-mini models show superior qualitative performance, while fine-tuned models excelled in accuracy and completeness but were less simple.
  arXiv  Detail & Related papers  (2025-02-19T23:07:16Z)
• Exploring ChatGPT for Face Presentation Attack Detection in Zero and Few-Shot in-Context Learning [6.537257913467247]
  This study highlights the potential of ChatGPT (specifically GPT-4o) as a competitive alternative for Face Presentation Attack Detection (PAD)<n>Our results show that GPT-4o demonstrates high consistency, particularly in few-shot in-context learning.<n>Remarkably, the model exhibits emergent reasoning capabilities, correctly predicting the attack type (print or replay) with high accuracy in few-shot scenarios.
  arXiv  Detail & Related papers  (2025-01-15T13:46:33Z)
• Chasing Fairness in Graphs: A GNN Architecture Perspective [73.43111851492593]
  We propose textsfFair textsfMessage textsfPassing (FMP) designed within a unified optimization framework for graph neural networks (GNNs) In FMP, the aggregation is first adopted to utilize neighbors' information and then the bias mitigation step explicitly pushes demographic group node presentation centers together. Experiments on node classification tasks demonstrate that the proposed FMP outperforms several baselines in terms of fairness and accuracy on three real-world datasets.
  arXiv  Detail & Related papers  (2023-12-19T18:00:15Z)
• CritiqueLLM: Towards an Informative Critique Generation Model for Evaluation of Large Language Model Generation [87.44350003888646]
  Eval-Instruct can acquire pointwise grading critiques with pseudo references and revise these critiques via multi-path prompting. CritiqueLLM is empirically shown to outperform ChatGPT and all the open-source baselines.
  arXiv  Detail & Related papers  (2023-11-30T16:52:42Z)
• Interpretable-by-Design Text Understanding with Iteratively Generated Concept Bottleneck [46.015128326688234]
  Black-box deep neural networks excel in text classification, yet their application in high-stakes domains is hindered by their lack of interpretability. We propose Text Bottleneck Models (TBM), an intrinsically interpretable text classification framework that offers both global and local explanations.
  arXiv  Detail & Related papers  (2023-10-30T15:41:32Z)
• Large Language Models for Propaganda Detection [2.587450057509126]
  This study investigates the effectiveness of Large Language Models (LLMs) for propaganda detection. Five variations of GPT-3 and GPT-4 are employed, incorporating various prompt engineering and fine-tuning strategies. Our findings demonstrate that GPT-4 achieves comparable results to the current state-of-the-art.
  arXiv  Detail & Related papers  (2023-10-10T08:46:10Z)
• Towards Reliable Misinformation Mitigation: Generalization, Uncertainty, and GPT-4 [5.313670352036673]
  We show that GPT-4 can outperform prior methods in multiple settings and languages. We propose techniques to handle uncertainty that can detect impossible examples and strongly improve outcomes. This research lays the groundwork for future tools that can drive real-world progress to combat misinformation.
  arXiv  Detail & Related papers  (2023-05-24T09:10:20Z)
• Can GPT-4 Perform Neural Architecture Search? [56.98363718371614]
  We investigate the potential of GPT-4 to perform Neural Architecture Search (NAS) Our proposed approach, textbfGPT-4 textbfEnhanced textbfNeural archtextbfItecttextbfUre textbfSearch (GENIUS) We assess GENIUS across several benchmarks, comparing it with existing state-of-the-art NAS techniques to illustrate its effectiveness.
  arXiv  Detail & Related papers  (2023-04-21T14:06:44Z)
• Patch-Prompt Aligned Bayesian Prompt Tuning for Vision-Language Models [48.77653835765705]
  We introduce a probabilistic resolution to prompt tuning, where the label-specific prompts are generated hierarchically by first sampling a latent vector from an underlying distribution and then employing a lightweight generative model. We evaluate the effectiveness of our approach on four tasks: few-shot image recognition, base-to-new generalization, dataset transfer learning, and domain shifts.
  arXiv  Detail & Related papers  (2023-03-16T06:09:15Z)
• Few-shot Text Classification with Dual Contrastive Consistency [31.141350717029358]
  In this paper, we explore how to utilize pre-trained language model to perform few-shot text classification. We adopt supervised contrastive learning on few labeled data and consistency-regularization on vast unlabeled data.
  arXiv  Detail & Related papers  (2022-09-29T19:26:23Z)
• News Summarization and Evaluation in the Era of GPT-3 [73.48220043216087]
  We study how GPT-3 compares against fine-tuned models trained on large summarization datasets. We show that not only do humans overwhelmingly prefer GPT-3 summaries, prompted using only a task description, but these also do not suffer from common dataset-specific issues such as poor factuality.
  arXiv  Detail & Related papers  (2022-09-26T01:04:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.