Distributed Asymmetric Allocation: A Topic Model for Large Imbalanced Corpora in Social Sciences

• URL: http://arxiv.org/abs/2512.18119v1
• Date: Fri, 19 Dec 2025 22:56:57 GMT
• Title: Distributed Asymmetric Allocation: A Topic Model for Large Imbalanced Corpora in Social Sciences
• Abstract summary: I develop a new topic model called distributed asymmetric allocation (DAA)<n>DAA integrates multiple algorithms for efficiently identifying sentences about important topics in large corpora.<n>I evaluate the ability of DAA to identify politically important topics by fitting it to the transcripts of speeches at the United Nations General Assembly between 1991 and 2017.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Social scientists employ latent Dirichlet allocation (LDA) to find highly specific topics in large corpora, but they often struggle in this task because (1) LDA, in general, takes a significant amount of time to fit on large corpora; (2) unsupervised LDA fragments topics into sub-topics in short documents; (3) semi-supervised LDA fails to identify specific topics defined using seed words. To solve these problems, I have developed a new topic model called distributed asymmetric allocation (DAA) that integrates multiple algorithms for efficiently identifying sentences about important topics in large corpora. I evaluate the ability of DAA to identify politically important topics by fitting it to the transcripts of speeches at the United Nations General Assembly between 1991 and 2017. The results show that DAA can classify sentences significantly more accurately and quickly than LDA thanks to the new algorithms. More generally, the results demonstrate that it is important for social scientists to optimize Dirichlet priors of LDA to perform content analysis accurately.

Related papers

• Quantifying consistency and accuracy of Latent Dirichlet Allocation [2.099922236065961]
  probabilistic topic models can produce different results when rerun due to their nature, leading to inconsistencies in latent topics.<n>This instability affects replicability, reliability, and interpretation, raising concerns about whether topic models capture meaningful topics or just noise.<n>We show that LDA can correctly determine the underlying number of topics in the documents.
  arXiv  Detail & Related papers  (2025-11-17T00:44:27Z)
• Analyzing Political Text at Scale with Online Tensor LDA [53.16930342547758]
  This paper proposes a topic modeling method that scales linearly to billions of documents.<n>We show that this method is computationally and memory efficient (achieving speeds over 3-4x those of prior parallelized Latent Dirichlet Allocation (LDA) methods)<n>We perform two real-world, large-scale new studies of interest to political scientists.
  arXiv  Detail & Related papers  (2025-11-11T03:58:48Z)
• Towards Agentic RAG with Deep Reasoning: A Survey of RAG-Reasoning Systems in LLMs [69.10441885629787]
  Retrieval-Augmented Generation (RAG) lifts the factuality of Large Language Models (LLMs) by injecting external knowledge.<n>It falls short on problems that demand multi-step inference; conversely, purely reasoning-oriented approaches often hallucinate or mis-ground facts.<n>This survey synthesizes both strands under a unified reasoning-retrieval perspective.
  arXiv  Detail & Related papers  (2025-07-13T03:29:41Z)
• What Constitutes a Less Discriminatory Algorithm? [2.842548870013324]
  We argue that formal LDA definitions face fundamental challenges when they attempt to evaluate and compare predictive models in the absence of held-out data.<n>We put forward a framework in which both firms and plaintiffs can search for alternative models that comport with societal goals.
  arXiv  Detail & Related papers  (2024-12-24T03:49:48Z)
• The LLM Effect: Are Humans Truly Using LLMs, or Are They Being Influenced By Them Instead? [60.01746782465275]
  Large Language Models (LLMs) have shown capabilities close to human performance in various analytical tasks. This paper investigates the efficiency and accuracy of LLMs in specialized tasks through a structured user study focusing on Human-LLM partnership.
  arXiv  Detail & Related papers  (2024-10-07T02:30:18Z)
• Large Multimodal Agents: A Survey [78.81459893884737]
  Large language models (LLMs) have achieved superior performance in powering text-based AI agents. There is an emerging research trend focused on extending these LLM-powered AI agents into the multimodal domain. This review aims to provide valuable insights and guidelines for future research in this rapidly evolving field.
  arXiv  Detail & Related papers  (2024-02-23T06:04:23Z)
• Algorithm-Dependent Bounds for Representation Learning of Multi-Source Domain Adaptation [7.6249291891777915]
  We use information-theoretic tools to derive a novel analysis of Multi-source Domain Adaptation (MDA) from the representation learning perspective. We propose a novel deep MDA algorithm, implicitly addressing the target shift through joint alignment. The proposed algorithm has comparable performance to the state-of-the-art on target-shifted MDA benchmark with improved memory efficiency.
  arXiv  Detail & Related papers  (2023-04-04T18:32:20Z)
• Goal Driven Discovery of Distributional Differences via Language Descriptions [58.764821647036946]
  Mining large corpora can generate useful discoveries but is time-consuming for humans. We formulate a new task, D5, that automatically discovers differences between two large corpora in a goal-driven way. Our system produces discoveries previously unknown to the authors on a wide range of applications in OpenD5.
  arXiv  Detail & Related papers  (2023-02-28T01:32:32Z)
• Full-Text Argumentation Mining on Scientific Publications [3.8754200816873787]
  We introduce a sequential pipeline model combining ADUR and ARE for full-text SAM. We provide a first analysis of the performance of pretrained language models (PLMs) on both subtasks. Our detailed error analysis reveals that non-contiguous ADUs as well as the interpretation of discourse connectors pose major challenges.
  arXiv  Detail & Related papers  (2022-10-24T10:05:30Z)
• EvaLDA: Efficient Evasion Attacks Towards Latent Dirichlet Allocation [9.277398460006394]
  We study whether Latent Dirichlet Allocation models are vulnerable to adversarial perturbations during inference time. We propose a novel and efficient algorithm, EvaLDA, to solve it. Our work provides significant insights into the power and limitations of evasion attacks to LDA models.
  arXiv  Detail & Related papers  (2020-12-09T04:57:20Z)
• Self-Weighted Robust LDA for Multiclass Classification with Edge Classes [111.5515086563592]
  A novel self-weighted robust LDA with l21-norm based between-class distance criterion, called SWRLDA, is proposed for multi-class classification. The proposed SWRLDA is easy to implement, and converges fast in practice.
  arXiv  Detail & Related papers  (2020-09-24T12:32:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.