Zero-shot narrative detection in social messaging

• URL: http://arxiv.org/abs/2609.17310v1
• Date: Tue, 15 Sep 2026 15:20:49 GMT
• Title: Zero-shot narrative detection in social messaging
• Abstract summary: This study investigates the zero-shot ability of large language models (LLMs) to identify and classify hidden narratives in social messages.<n>Our research hypothesis is that LLMs' extensive contextual knowledge allows them to interpret messages on a deeper, pragmatic level.<n>Experiments on the Dipromats and SemEval datasets show that providing models with human-written narrative descriptions significantly improves performance.
• Score: 0.688204255655161
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This study investigates the zero-shot ability of large language models (LLMs) to identify and classify hidden narratives in social messages. Our research hypothesis is that LLMs' extensive contextual knowledge allows them to interpret messages on a deeper, pragmatic level, going beyond basic sentiment or topic analysis. Experiments on the Dipromats and SemEval datasets show that providing models with human-written narrative descriptions significantly improves performance, without the need of training examples. In contrast, automatically generated descriptions or the use of few examples (few-shot) often degrade accuracy due to subtle shifts in framing. The study also finds that ensemble methods, particularly majority voting, enhance robustness and that larger models perform best while also being less sensitive to prompt variations. The findings validate that LLMs can effectively detect strategic narratives in a zero-shot setting, and when combined with simple ensembling and human-written descriptions, they can rival supervised systems, offering a scalable solution for narrative detection, specially when there is no training data for the vast majority of domains.

Related papers

• Farther the Shift, Sparser the Representation: Analyzing OOD Mechanisms in LLMs [100.02824137397464]
  We investigate how Large Language Models adapt their internal representations when encountering inputs of increasing difficulty.<n>We reveal a consistent and quantifiable phenomenon: as task difficulty increases, the last hidden states of LLMs become substantially sparser.<n>This sparsity--difficulty relation is observable across diverse models and domains.
  arXiv  Detail & Related papers  (2026-03-03T18:48:15Z)
• A Multilingual, Large-Scale Study of the Interplay between LLM Safeguards, Personalisation, and Disinformation [12.577461004484604]
  This study presents the first large-scale, multilingual analysis of persona-targeted disinformation generation by Large Language Models (LLMs)<n>We prompt eight state-of-the-art LLMs with 324 false narratives and 150 demographic personas across four languages--English, Russian, Portuguese, and Hindi--resulting in AI-TRAITS.<n>Results show that the use of even simple personalisation prompts significantly increases the likelihood of jailbreaks.
  arXiv  Detail & Related papers  (2025-10-14T21:10:50Z)
• Improving Narrative Classification and Explanation via Fine Tuned Language Models [6.310433217813068]
  This study tackles two key challenges: (1) multi-label classification of narratives and sub-narratives in news articles, and (2) generating concise, evidence-based explanations for dominant narratives.<n>We fine-tune a BERT model with a recall-oriented approach for comprehensive narrative detection, refining predictions using a GPT-4o pipeline for consistency.<n>Our results show that integrating auxiliary knowledge in prompts improves classification accuracy and justification reliability, with applications in media analysis, education, and intelligence gathering.
  arXiv  Detail & Related papers  (2025-09-04T10:12:31Z)
• Bridging the Gap: In-Context Learning for Modeling Human Disagreement [8.011316959982654]
  Large Language Models (LLMs) have shown strong performance on NLP classification tasks.<n>This study examines whether LLMs can capture multiple perspectives and reflect annotator disagreement in subjective tasks such as hate speech and offensive language detection.
  arXiv  Detail & Related papers  (2025-06-06T14:24:29Z)
• Contextualize-then-Aggregate: Circuits for In-Context Learning in Gemma-2 2B [51.74607395697567]
  In-Context Learning (ICL) is an intriguing ability of large language models (LLMs)<n>We use causal interventions to identify information flow in Gemma-2 2B for five naturalistic ICL tasks.<n>We find that the model infers task information using a two-step strategy we call contextualize-then-aggregate.
  arXiv  Detail & Related papers  (2025-03-31T18:33:55Z)
• MIR-Bench: Can Your LLM Recognize Complicated Patterns via Many-Shot In-Context Reasoning? [21.638848019633595]
  We propose MIR-Bench, the first many-shot in-context reasoning benchmark for pattern recognition.<n>We study many novel problems for many-shot in-context reasoning, and acquired many insightful findings.
  arXiv  Detail & Related papers  (2025-02-14T06:05:12Z)
• Explingo: Explaining AI Predictions using Large Language Models [47.21393184176602]
  Large Language Models (LLMs) can transform explanations into human-readable, narrative formats that align with natural communication.<n>The Narrator takes in ML explanations and transforms them into natural-language descriptions.<n>The Grader scores these narratives on a set of metrics including accuracy, completeness, fluency, and conciseness.<n>The findings from this work have been integrated into an open-source tool that makes narrative explanations available for further applications.
  arXiv  Detail & Related papers  (2024-12-06T16:01:30Z)
• Narrative Analysis of True Crime Podcasts With Knowledge Graph-Augmented Large Language Models [8.78598447041169]
  Large language models (LLMs) still struggle with complex narrative arcs as well as narratives containing conflicting information. Recent work indicates LLMs augmented with external knowledge bases can improve the accuracy and interpretability of the resulting models. In this work, we analyze the effectiveness of applying knowledge graphs (KGs) in understanding true-crime podcast data.
  arXiv  Detail & Related papers  (2024-11-01T21:49:00Z)
• C-ICL: Contrastive In-context Learning for Information Extraction [54.39470114243744]
  c-ICL is a novel few-shot technique that leverages both correct and incorrect sample constructions to create in-context learning demonstrations. Our experiments on various datasets indicate that c-ICL outperforms previous few-shot in-context learning methods.
  arXiv  Detail & Related papers  (2024-02-17T11:28:08Z)
• Coverage-based Example Selection for In-Context Learning [27.215972147196805]
  We show that BERTScore-Recall (BSR) selects better examples that demonstrate more of the salient aspects of the test input. On 15 datasets spanning 6 tasks and with 7 diverse LLMs, we show that (1) BSR is the superior metric for in-context example selection across the board, and (2) for compositional tasks, Set-BSR outperforms independent ranking by up to 17 points on average.
  arXiv  Detail & Related papers  (2023-05-24T08:58:28Z)
• Large Language Models Are Latent Variable Models: Explaining and Finding Good Demonstrations for In-Context Learning [104.58874584354787]
  In recent years, pre-trained large language models (LLMs) have demonstrated remarkable efficiency in achieving an inference-time few-shot learning capability known as in-context learning. This study aims to examine the in-context learning phenomenon through a Bayesian lens, viewing real-world LLMs as latent variable models.
  arXiv  Detail & Related papers  (2023-01-27T18:59:01Z)
• Few-shot Instruction Prompts for Pretrained Language Models to Detect Social Biases [55.45617404586874]
  We propose a few-shot instruction-based method for prompting pre-trained language models (LMs) We show that large LMs can detect different types of fine-grained biases with similar and sometimes superior accuracy to fine-tuned models.
  arXiv  Detail & Related papers  (2021-12-15T04:19:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.