Identifying economic narratives in large text corpora -- An integrated approach using Large Language Models

• URL: http://arxiv.org/abs/2506.15041v1
• Date: Wed, 18 Jun 2025 01:00:59 GMT
• Title: Identifying economic narratives in large text corpora -- An integrated approach using Large Language Models
• Authors: Tobias Schmidt, Kai-Robin Lange, Matthias Reccius, Henrik Müller, Michael Roos, Carsten Jentsch,
• Abstract summary: We evaluate the benefits of Large Language Models (LLMs) for extracting economic narratives from texts.<n>We apply a rigorous narrative definition and compare GPT-4o outputs to gold-standard narratives produced by expert annotators.<n>Our results suggest GPT-4o is capable of extracting valid economic narratives in a structured format, but still falls short of expert-level performance when handling complex documents and narratives.
• Score: 0.4649452333875421
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As interest in economic narratives has grown in recent years, so has the number of pipelines dedicated to extracting such narratives from texts. Pipelines often employ a mix of state-of-the-art natural language processing techniques, such as BERT, to tackle this task. While effective on foundational linguistic operations essential for narrative extraction, such models lack the deeper semantic understanding required to distinguish extracting economic narratives from merely conducting classic tasks like Semantic Role Labeling. Instead of relying on complex model pipelines, we evaluate the benefits of Large Language Models (LLMs) by analyzing a corpus of Wall Street Journal and New York Times newspaper articles about inflation. We apply a rigorous narrative definition and compare GPT-4o outputs to gold-standard narratives produced by expert annotators. Our results suggests that GPT-4o is capable of extracting valid economic narratives in a structured format, but still falls short of expert-level performance when handling complex documents and narratives. Given the novelty of LLMs in economic research, we also provide guidance for future work in economics and the social sciences that employs LLMs to pursue similar objectives.

Related papers

• Narrative Shift Detection: A Hybrid Approach of Dynamic Topic Models and Large Language Models [0.4649452333875421]
  We propose a combination of the language understanding capabilities of Large Language Models with the large scale applicability of topic models to dynamically model narrative shifts across time.<n>We employ our pipeline on a corpus of The Wall Street Journal news paper articles from 2009 to 2023.
  arXiv  Detail & Related papers  (2025-06-25T09:25:15Z)
• Explingo: Explaining AI Predictions using Large Language Models [47.21393184176602]
  Large Language Models (LLMs) can transform explanations into human-readable, narrative formats that align with natural communication.<n>The Narrator takes in ML explanations and transforms them into natural-language descriptions.<n>The Grader scores these narratives on a set of metrics including accuracy, completeness, fluency, and conciseness.<n>The findings from this work have been integrated into an open-source tool that makes narrative explanations available for further applications.
  arXiv  Detail & Related papers  (2024-12-06T16:01:30Z)
• Narrative Analysis of True Crime Podcasts With Knowledge Graph-Augmented Large Language Models [8.78598447041169]
  Large language models (LLMs) still struggle with complex narrative arcs as well as narratives containing conflicting information. Recent work indicates LLMs augmented with external knowledge bases can improve the accuracy and interpretability of the resulting models. In this work, we analyze the effectiveness of applying knowledge graphs (KGs) in understanding true-crime podcast data.
  arXiv  Detail & Related papers  (2024-11-01T21:49:00Z)
• Mapping News Narratives Using LLMs and Narrative-Structured Text Embeddings [0.0]
  We introduce a numerical narrative representation grounded in structuralist linguistic theory. We extract the actants using an open-source LLM and integrate them into a Narrative-Structured Text Embedding. We demonstrate the analytical insights of the method on the example of 5000 full-text news articles from Al Jazeera and The Washington Post on the Israel-Palestine conflict.
  arXiv  Detail & Related papers  (2024-09-10T14:15:30Z)
• Are Large Language Models Capable of Generating Human-Level Narratives? [114.34140090869175]
  This paper investigates the capability of LLMs in storytelling, focusing on narrative development and plot progression. We introduce a novel computational framework to analyze narratives through three discourse-level aspects. We show that explicit integration of discourse features can enhance storytelling, as is demonstrated by over 40% improvement in neural storytelling.
  arXiv  Detail & Related papers  (2024-07-18T08:02:49Z)
• Can LLMs Learn Macroeconomic Narratives from Social Media? [20.11321226491191]
  We introduce two curated datasets containing posts from X (formerly Twitter) which capture economy-related narratives.<n>We extract and summarize narratives from the tweets using Natural Language Processing (NLP) methods.<n>We test their predictive power for $textitmacroeconomic$ forecasting by incorporating the tweets' or the extracted narratives' representations in downstream financial prediction tasks.
  arXiv  Detail & Related papers  (2024-06-17T21:37:09Z)
• Can LLMs Understand the Implication of Emphasized Sentences in Dialogue? [64.72966061510375]
  Emphasis is a crucial component in human communication, which indicates the speaker's intention and implication beyond pure text in dialogue. This paper introduces Emphasized-Talk, a benchmark with emphasis-annotated dialogue samples capturing the implications of emphasis. We evaluate various Large Language Models (LLMs), both open-source and commercial, to measure their performance in understanding emphasis.
  arXiv  Detail & Related papers  (2024-06-16T20:41:44Z)
• GPT-4 Generated Narratives of Life Events using a Structured Narrative Prompt: A Validation Study [0.0]
  We employ a zero-shot structured narrative prompt to generate 24,000 narratives using OpenAI's GPT-4. From this dataset, we manually classify 2,880 narratives and evaluate their validity in conveying birth, death, hiring, and firing events. We extend our analysis to predict the classifications of the remaining 21,120 narratives.
  arXiv  Detail & Related papers  (2024-02-08T06:20:01Z)
• Mastering the Task of Open Information Extraction with Large Language Models and Consistent Reasoning Environment [52.592199835286394]
  Open Information Extraction (OIE) aims to extract objective structured knowledge from natural texts. Large language models (LLMs) have exhibited remarkable in-context learning capabilities.
  arXiv  Detail & Related papers  (2023-10-16T17:11:42Z)
• Enhancing Argument Structure Extraction with Efficient Leverage of Contextual Information [79.06082391992545]
  We propose an Efficient Context-aware model (ECASE) that fully exploits contextual information. We introduce a sequence-attention module and distance-weighted similarity loss to aggregate contextual information and argumentative information. Our experiments on five datasets from various domains demonstrate that our model achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-10-08T08:47:10Z)
• Document-Level Machine Translation with Large Language Models [91.03359121149595]
  Large language models (LLMs) can produce coherent, cohesive, relevant, and fluent answers for various natural language processing (NLP) tasks. This paper provides an in-depth evaluation of LLMs' ability on discourse modeling.
  arXiv  Detail & Related papers  (2023-04-05T03:49:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.