Exploiting contextual information to improve stance detection in informal political discourse with LLMs

• URL: http://arxiv.org/abs/2602.04750v1
• Date: Wed, 04 Feb 2026 16:49:26 GMT
• Title: Exploiting contextual information to improve stance detection in informal political discourse with LLMs
• Authors: Arman Engin Sucu, Yixiang Zhou, Mario A. Nascimento, Tony Mullen,
• Abstract summary: This study investigates the use of Large Language Models (LLMs) for political stance detection in informal online discourse.<n>Using a real-world political forum dataset, we generate structured profiles that summarize users' ideological leaning, recurring topics, and linguistic patterns.<n>We show that contextual prompts significantly boost accuracy, with improvements ranging from +17.5% to +38.5%, achieving up to 74% accuracy that surpasses previous approaches.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This study investigates the use of Large Language Models (LLMs) for political stance detection in informal online discourse, where language is often sarcastic, ambiguous, and context-dependent. We explore whether providing contextual information, specifically user profile summaries derived from historical posts, can improve classification accuracy. Using a real-world political forum dataset, we generate structured profiles that summarize users' ideological leaning, recurring topics, and linguistic patterns. We evaluate seven state-of-the-art LLMs across baseline and context-enriched setups through a comprehensive cross-model evaluation. Our findings show that contextual prompts significantly boost accuracy, with improvements ranging from +17.5\% to +38.5\%, achieving up to 74\% accuracy that surpasses previous approaches. We also analyze how profile size and post selection strategies affect performance, showing that strategically chosen political content yields better results than larger, randomly selected contexts. These findings underscore the value of incorporating user-level context to enhance LLM performance in nuanced political classification tasks.

Related papers

• Beyond Naïve Prompting: Strategies for Improved Zero-shot Context-aided Forecasting with LLMs [57.82819770709032]
  Large language models (LLMs) can be effective context-aided forecasters via na"ive direct prompting.<n>ReDP improves interpretability by eliciting explicit reasoning traces, allowing us to assess the model's reasoning over the context.<n>CorDP leverages LLMs solely to refine existing forecasts with context, enhancing their applicability in real-world forecasting pipelines.<n> IC-DP proposes embedding historical examples of context-aided forecasting tasks in the prompt, substantially improving accuracy even for the largest models.
  arXiv  Detail & Related papers  (2025-08-13T16:02:55Z)
• AgoraSpeech: A multi-annotated comprehensive dataset of political discourse through the lens of humans and AI [1.3060410279656598]
  AgoraSpeech is a meticulously curated, high-quality dataset of 171 political speeches from six parties during the Greek national elections in 2023.<n>The dataset includes annotations (per paragraph) for six natural language processing (NLP) tasks: text classification, topic identification, sentiment analysis, named entity recognition, polarization and populism detection.
  arXiv  Detail & Related papers  (2025-01-09T18:17:59Z)
• Context-DPO: Aligning Language Models for Context-Faithfulness [80.62221491884353]
  We propose the first alignment method specifically designed to enhance large language models' context-faithfulness.<n>By leveraging faithful and stubborn responses to questions with provided context from ConFiQA, our Context-DPO aligns LLMs through direct preference optimization.<n>Extensive experiments demonstrate that our Context-DPO significantly improves context-faithfulness, achieving 35% to 280% improvements on popular open-source models.
  arXiv  Detail & Related papers  (2024-12-18T04:08:18Z)
• Align-SLM: Textless Spoken Language Models with Reinforcement Learning from AI Feedback [50.84142264245052]
  This work introduces the Align-SLM framework to enhance the semantic understanding of textless Spoken Language Models (SLMs)<n>Our approach generates multiple speech continuations from a given prompt and uses semantic metrics to create preference data for Direct Preference Optimization (DPO)<n>We evaluate the framework using ZeroSpeech 2021 benchmarks for lexical and syntactic modeling, the spoken version of the StoryCloze dataset for semantic coherence, and other speech generation metrics, including the GPT4-o score and human evaluation.
  arXiv  Detail & Related papers  (2024-11-04T06:07:53Z)
• Language Models Learn Metadata: Political Stance Detection Case Study [1.2277343096128712]
  This paper investigates the optimal way to incorporate metadata into a political stance detection task. We show that our simple baseline, using only party membership information, surpasses the current state-of-the-art.
  arXiv  Detail & Related papers  (2024-09-15T14:57:41Z)
• Con-ReCall: Detecting Pre-training Data in LLMs via Contrastive Decoding [118.75567341513897]
  Existing methods typically analyze target text in isolation or solely with non-member contexts.<n>We propose Con-ReCall, a novel approach that leverages the asymmetric distributional shifts induced by member and non-member contexts.
  arXiv  Detail & Related papers  (2024-09-05T09:10:38Z)
• Examining the Influence of Political Bias on Large Language Model Performance in Stance Classification [5.8229466650067065]
  We investigate whether large language models (LLMs) exhibit a tendency to more accurately classify politically-charged stances. Our findings reveal a statistically significant difference in the performance of LLMs across various politically oriented stance classification tasks. LLMs have poorer stance classification accuracy when there is greater ambiguity in the target the statement is directed towards.
  arXiv  Detail & Related papers  (2024-07-25T01:11:38Z)
• PoliTune: Analyzing the Impact of Data Selection and Fine-Tuning on Economic and Political Biases in Large Language Models [1.1704154007740835]
  This work investigates the impact of fine-tuning and data selection on economic and political biases in Large Language Models (LLMs) We introduce PoliTune, a fine-tuning methodology to explore the systematic aspects of aligning LLMs with specific ideologies. We introduce a systematic method for using the open-source Llama3-70B for dataset selection, annotation, and synthesizing a preferences dataset for Direct Preference Optimization (DPO) to align the model with a given political ideology.
  arXiv  Detail & Related papers  (2024-04-10T16:30:09Z)
• Whose Side Are You On? Investigating the Political Stance of Large Language Models [56.883423489203786]
  We investigate the political orientation of Large Language Models (LLMs) across a spectrum of eight polarizing topics. Our investigation delves into the political alignment of LLMs across a spectrum of eight polarizing topics, spanning from abortion to LGBTQ issues. The findings suggest that users should be mindful when crafting queries, and exercise caution in selecting neutral prompt language.
  arXiv  Detail & Related papers  (2024-03-15T04:02:24Z)
• Generative Context-aware Fine-tuning of Self-supervised Speech Models [54.389711404209415]
  We study the use of generative large language models (LLM) generated context information. We propose an approach to distill the generated information during fine-tuning of self-supervised speech models. We evaluate the proposed approach using the SLUE and Libri-light benchmarks for several downstream tasks: automatic speech recognition, named entity recognition, and sentiment analysis.
  arXiv  Detail & Related papers  (2023-12-15T15:46:02Z)
• Exploring the Jungle of Bias: Political Bias Attribution in Language Models via Dependency Analysis [86.49858739347412]
  Large Language Models (LLMs) have sparked intense debate regarding the prevalence of bias in these models and its mitigation. We propose a prompt-based method for the extraction of confounding and mediating attributes which contribute to the decision process. We find that the observed disparate treatment can at least in part be attributed to confounding and mitigating attributes and model misalignment.
  arXiv  Detail & Related papers  (2023-11-15T00:02:25Z)
• Reference-less Analysis of Context Specificity in Translation with Personalised Language Models [3.527589066359829]
  This work investigates what extent rich character and film annotations can be leveraged to personalise language models (LMs) We build LMs which leverage rich contextual information to reduce perplexity by up to 6.5% compared to a non-contextual model. Our results suggest that the degree to which professional translations in our domain are context-specific can be preserved to a better extent by a contextual machine translation model.
  arXiv  Detail & Related papers  (2023-03-29T12:19:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.