Assessing the Applicability of Natural Language Processing to Traditional Social Science Methodology: A Case Study in Identifying Strategic Signaling Patterns in Presidential Directives

• URL: http://arxiv.org/abs/2511.09738v1
• Date: Fri, 14 Nov 2025 01:07:12 GMT
• Title: Assessing the Applicability of Natural Language Processing to Traditional Social Science Methodology: A Case Study in Identifying Strategic Signaling Patterns in Presidential Directives
• Authors: C. LeMay, A. Lane, J. Seales, M. Winstead, S. Baty,
• Abstract summary: This research investigates how Natural Language Processing (NLP) can be used to extract main topics from a larger corpus of written data.<n>Analysts and NLP both identified relevant documents, demonstrating the potential utility of NLPs in research involving large written corpuses.<n>We also identified discrepancies between NLP and human-labeled results that indicate a need for more research to assess the validity of NLP in this use case.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Our research investigates how Natural Language Processing (NLP) can be used to extract main topics from a larger corpus of written data, as applied to the case of identifying signaling themes in Presidential Directives (PDs) from the Reagan through Clinton administrations. Analysts and NLP both identified relevant documents, demonstrating the potential utility of NLPs in research involving large written corpuses. However, we also identified discrepancies between NLP and human-labeled results that indicate a need for more research to assess the validity of NLP in this use case. The research was conducted in 2023, and the rapidly evolving landscape of AIML means existing tools have improved and new tools have been developed; this research displays the inherent capabilities of a potentially dated AI tool in emerging social science applications.

Related papers

• Social Good or Scientific Curiosity? Uncovering the Research Framing Behind NLP Artefacts [11.383081780085321]
  Clarifying the research framing of NLP artefacts is crucial to aligning research with practical applications.<n>Recent studies manually analyzed NLP research across domains, showing that few papers explicitly identify key stakeholders, intended uses, or appropriate contexts.<n>We develop a three-component system that infers research framings by first extracting key elements (means, ends, stakeholders), then linking them through interpretable rules and contextual reasoning.
  arXiv  Detail & Related papers  (2025-05-24T12:46:26Z)
• The Nature of NLP: Analyzing Contributions in NLP Papers [77.31665252336157]
  We propose a taxonomy of research contributions and introduce NLPContributions, a dataset of nearly $2k$ NLP research paper abstracts.<n>We show that NLP research has taken a winding path -- with the focus on language and human-centric studies being prominent in the 1970s and 80s, tapering off in the 1990s and 2000s, and starting to rise again since the late 2010s.<n>Our dataset and analyses offer a powerful lens for tracing research trends and offer potential for generating informed, data-driven literature surveys.
  arXiv  Detail & Related papers  (2024-09-29T01:29:28Z)
• Emergent Language: A Survey and Taxonomy [9.823821010022932]
  The paper provides a comprehensive review of 181 scientific publications on emergent language in artificial intelligence.<n>Its objective is to serve as a reference for researchers interested in or proficient in the field.
  arXiv  Detail & Related papers  (2024-09-04T12:22:05Z)
• From Insights to Actions: The Impact of Interpretability and Analysis Research on NLP [28.942812379900673]
  Interpretability and analysis (IA) research is a growing subfield within NLP. We seek to quantify the impact of IA research on the broader field of NLP.
  arXiv  Detail & Related papers  (2024-06-18T13:45:07Z)
• Natural Language Processing for Dialects of a Language: A Survey [56.93337350526933]
  State-of-the-art natural language processing (NLP) models are trained on massive training corpora, and report a superlative performance on evaluation datasets.<n>This survey delves into an important attribute of these datasets: the dialect of a language.<n>Motivated by the performance degradation of NLP models for dialectal datasets and its implications for the equity of language technologies, we survey past research in NLP for dialects in terms of datasets, and approaches.
  arXiv  Detail & Related papers  (2024-01-11T03:04:38Z)
• Exploring the Landscape of Natural Language Processing Research [3.3916160303055567]
  Several NLP-related approaches have been surveyed in the research community. A comprehensive study that categorizes established topics, identifies trends, and outlines areas for future research remains absent. As a result, we present a structured overview of the research landscape, provide a taxonomy of fields of study in NLP, analyze recent developments in NLP, summarize our findings, and highlight directions for future work.
  arXiv  Detail & Related papers  (2023-07-20T07:33:30Z)
• Application of Transformers based methods in Electronic Medical Records: A Systematic Literature Review [77.34726150561087]
  This work presents a systematic literature review of state-of-the-art advances using transformer-based methods on electronic medical records (EMRs) in different NLP tasks.
  arXiv  Detail & Related papers  (2023-04-05T22:19:42Z)
• An Inclusive Notion of Text [69.36678873492373]
  We argue that clarity on the notion of text is crucial for reproducible and generalizable NLP. We introduce a two-tier taxonomy of linguistic and non-linguistic elements that are available in textual sources and can be used in NLP modeling.
  arXiv  Detail & Related papers  (2022-11-10T14:26:43Z)
• Systematic Inequalities in Language Technology Performance across the World's Languages [94.65681336393425]
  We introduce a framework for estimating the global utility of language technologies. Our analyses involve the field at large, but also more in-depth studies on both user-facing technologies and more linguistic NLP tasks.
  arXiv  Detail & Related papers  (2021-10-13T14:03:07Z)
• Ensuring the Inclusive Use of Natural Language Processing in the Global Response to COVID-19 [58.720142291102135]
  We discuss ways in which current and future NLP approaches can be made more inclusive by covering low-resource languages. We suggest several future directions for researchers interested in maximizing the positive societal impacts of NLP.
  arXiv  Detail & Related papers  (2021-08-11T12:54:26Z)
• Natural Language Processing Advancements By Deep Learning: A Survey [0.755972004983746]
  This survey categorizes and addresses the different aspects and applications of NLP that have benefited from deep learning. It covers core NLP tasks and applications and describes how deep learning methods and models advance these areas.
  arXiv  Detail & Related papers  (2020-03-02T21:32:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.