Learning to Aggregate Zero-Shot LLM Agents for Corporate Disclosure Classification

• URL: http://arxiv.org/abs/2603.20965v1
• Date: Sat, 21 Mar 2026 22:29:19 GMT
• Title: Learning to Aggregate Zero-Shot LLM Agents for Corporate Disclosure Classification
• Abstract summary: This paper studies whether a lightweight trained aggregator can combine diverse zero-shot large language model judgments into a stronger downstream signal for corporate disclosure classification.<n>I use a sample of 18,420 U.S. corporate disclosures issued by Nasdaq and S&P 500 firms between 2018 and 2024, matched to next-day stock returns.<n>Results show that the trained aggregator outperforms all single agents, majority vote, confidence-weighted voting, and a FinBERT baseline.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper studies whether a lightweight trained aggregator can combine diverse zero-shot large language model judgments into a stronger downstream signal for corporate disclosure classification. Zero-shot LLMs can read disclosures without task-specific fine-tuning, but their predictions often vary across prompts, reasoning styles, and model families. I address this problem with a multi-agent framework in which three zero-shot agents independently read each disclosure and output a sentiment label, a confidence score, and a short rationale. A logistic meta-classifier then aggregates these signals to predict next-day stock return direction. I use a sample of 18,420 U.S. corporate disclosures issued by Nasdaq and S&P 500 firms between 2018 and 2024, matched to next-day stock returns. Results show that the trained aggregator outperforms all single agents, majority vote, confidence-weighted voting, and a FinBERT baseline. Balanced accuracy rises from 0.561 for the best single agent to 0.612 for the trained aggregator, with the largest gains in disclosures combining strong current performance with weak guidance or elevated risk. The results suggest that zero-shot LLM agents capture complementary financial signals and that supervised aggregation can turn cross-agent disagreement into a more useful classification target.

Related papers

• PACE: A Proxy for Agentic Capability Evaluation [60.3743414796937]
  We introduce PACE, a framework that constructs proxy benchmarks by selecting instances from existing non-agentic evaluations.<n>Experiments across 14 models, 4 agentic benchmarks, and 19 non-agentic benchmarks show that PACE-Bench predicts agentic scores with leave-one-out cross-validation (LOOCV) mean absolute error (MAE) under 4%, Spearman correlation above 0.80, and pairwise model-ranking accuracy around 85%, all at much less than 1% of the full agentic evaluation cost.
  arXiv  Detail & Related papers  (2026-07-02T10:59:03Z)
• Attributing Emergence in Million-Agent Systems [68.53670424791751]
  Large language models (LLMs) can simulate human-like reasoning and decision-making in individual agents.<n>Such studies require attributing macro emergence to individual agents.<n>We adapt Aumann--Shapley path-integral attribution to LLM-powered MAS at million-agent scale.
  arXiv  Detail & Related papers  (2026-05-12T01:49:41Z)
• Zero-Shot Confidence Estimation for Small LLMs: When Supervised Baselines Aren't Worth Training [0.0]
  RouteLLM-style supervised baselines are compared across three 7-8B model families and two datasets.<n>Average token log-probability, which requires no training data, matches or exceeds supervised baselines in-distribution.<n>Retrieval-conditional self-assessment, a pre-generation signal, improves over bare self-assessment by up to +0.069 AUROC at 3-10x lower latency than log-probability.
  arXiv  Detail & Related papers  (2026-05-04T05:33:03Z)
• Foresight Arena: An On-Chain Benchmark for Evaluating AI Forecasting Agents [0.0]
  We introduce Foresight Arena, the first permissionless, on-chain benchmark for evaluating AI forecasting agents on real-world prediction markets.<n>Performance is measured by the Brier Score and a novel Alpha Score -- proper scoring rules that incentivize honest probability reporting.<n>We show that detecting a true edge of $* = 0.02$ at 80% power requires approximately 350 resolved binary predictions.
  arXiv  Detail & Related papers  (2026-05-01T05:33:10Z)
• Signal or Noise in Multi-Agent LLM-based Stock Recommendations? [0.0]
  We present the first portfolio-level validation of MarketSenseAI, a deployed multi-agent LLM equity system.<n>The system routes four specialist agents through a agent that issues a monthly equity thesis and synthesis for each stock in its coverage universe.<n>We ask two questions: do its buy recommendations add value over both passive benchmarks and random selection, and what does the internal agent structure reveal about the source of the edge?
  arXiv  Detail & Related papers  (2026-04-19T08:43:31Z)
• LLM Agents Predict Social Media Reactions but Do Not Outperform Text Classifiers: Benchmarking Simulation Accuracy Using 120K+ Personas of 1511 Humans [36.156330303795016]
  This study benchmarks accuracy in predicting human social media reactions (like, dislike, comment, share, no reaction) across 120,000+ unique agent-persona combinations.<n>Agents achieved 70.7% overall accuracy, with LLM choice producing a 13 percentage-point performance spread.<n>The genuine predictive validity of zero-shot persona-prompted agents warns against potential manipulation through easily deploying swarms of behaviorally distinct AI agents on social media.
  arXiv  Detail & Related papers  (2026-03-31T19:27:59Z)
• FIPO: Eliciting Deep Reasoning with Future-KL Influenced Policy Optimization [84.58281577727566]
  We present Future-KL Influenced Policy Optimization (FIPO), a reinforcement learning algorithm designed to overcome reasoning bottlenecks in large language models.<n>FIPO addresses this by incorporating discounted future-KL divergence into the policy update, creating a dense advantage formulation that re-weights tokens based on their influence on subsequent trajectory behavior.<n> evaluated on Qwen2.5-32B, FIPO extends the average chain-of-thought length from roughly 4,000 to over 10,000 tokens and increases AIME 2024 Pass@1 accuracy from 50.0% to a peak of 58.0%.
  arXiv  Detail & Related papers  (2026-03-20T10:24:50Z)
• Can Blindfolded LLMs Still Trade? An Anonymization-First Framework for Portfolio Optimization [1.0957528713294875]
  LLM trading agents must demonstrate understanding of market dynamics rather than exploitation of memorized ticker associations.<n>We address two sources of spurious performance: memorization bias from ticker-specific pre-training, and survivorship bias from flawed backtesting.<n>Our approach is to blindfold the agents--anonymizing all identifiers--and verify whether meaningful signals persist.
  arXiv  Detail & Related papers  (2026-03-18T13:09:11Z)
• From Stochastic Answers to Verifiable Reasoning: Interpretable Decision-Making with LLM-Generated Code [0.0]
  Large language models (LLMs) are increasingly used for high-stakes decisionmaking.<n>Black-box models obscure their reasoning, while recent LLM-based rule systems rely on per-sample evaluation.<n>We propose reframing LLMs as code generators rather than per-instance evaluators.
  arXiv  Detail & Related papers  (2026-02-28T00:27:29Z)
• Benchmarking Zero-Shot Reasoning Approaches for Error Detection in Solidity Smart Contracts [0.0]
  This paper evaluates state-of-the-art LLMs on Solidity smart contract analysis using a balanced dataset of 400 contracts.<n>Models are evaluated using zero-shot prompting strategies, including zero-shot, zero-shot Chain-of-Thought (CoT) and zero-shot Tree-of-Thought (ToT)
  arXiv  Detail & Related papers  (2026-02-17T18:08:56Z)
• ArenaRL: Scaling RL for Open-Ended Agents via Tournament-based Relative Ranking [84.07076200941474]
  ArenaRL is a reinforcement learning paradigm that shifts from pointwise scalar scoring to intra-group relative ranking.<n>We construct an intra-group adversarial arena and devise a tournament-based ranking scheme to obtain stable advantage signals.<n>Experiments show that ArenaRL substantially outperforms standard RL baselines.
  arXiv  Detail & Related papers  (2026-01-10T08:43:07Z)
• Shoot First, Ask Questions Later? Building Rational Agents that Explore and Act Like People [81.63702981397408]
  Given limited resources, to what extent do agents based on language models (LMs) act rationally?<n>We develop methods to benchmark and enhance agentic information-seeking, drawing on insights from human behavior.<n>For Spotter agents, our approach boosts accuracy by up to 14.7% absolute over LM-only baselines; for Captain agents, it raises expected information gain (EIG) by up to 0.227 bits (94.2% of the achievable noise ceiling)
  arXiv  Detail & Related papers  (2025-10-23T17:57:28Z)
• Event-Aware Sentiment Factors from LLM-Augmented Financial Tweets: A Transparent Framework for Interpretable Quant Trading [0.0]
  This study showcases the unique utility of large language models (LLMs) in financial semantic annotation and alpha signal discovery.<n>We use an LLM to automatically assign multi-label event categories to high-sentiment-intensity tweets.<n>Experiments reveal that certain event labels consistently yield negative alpha, with Sharpe ratios as low as -0.38 and information coefficients exceeding 0.05.
  arXiv  Detail & Related papers  (2025-08-10T16:09:14Z)
• A Large-Scale Study of Relevance Assessments with Large Language Models: An Initial Look [52.114284476700874]
  This paper reports on the results of a large-scale evaluation (the TREC 2024 RAG Track) where four different relevance assessment approaches were deployed. We find that automatically generated UMBRELA judgments can replace fully manual judgments to accurately capture run-level effectiveness. Surprisingly, we find that LLM assistance does not appear to increase correlation with fully manual assessments, suggesting that costs associated with human-in-the-loop processes do not bring obvious tangible benefits.
  arXiv  Detail & Related papers  (2024-11-13T01:12:35Z)
• A Strategy to Combine 1stGen Transformers and Open LLMs for Automatic Text Classification [7.303409990401068]
  Large Language Models (LLMs) are considered the cutting edge in several NLP tasks. This study compares three 1stTRs with two open LLMs across 11 sentiment analysis datasets. We propose a strategy that seamlessly integrates 1stTRs with open LLMs based on prediction certainty.
  arXiv  Detail & Related papers  (2024-08-19T01:22:21Z)
• Cycles of Thought: Measuring LLM Confidence through Stable Explanations [53.15438489398938]
  Large language models (LLMs) can reach and even surpass human-level accuracy on a variety of benchmarks, but their overconfidence in incorrect responses is still a well-documented failure mode. We propose a framework for measuring an LLM's uncertainty with respect to the distribution of generated explanations for an answer.
  arXiv  Detail & Related papers  (2024-06-05T16:35:30Z)
• Agent-FLAN: Designing Data and Methods of Effective Agent Tuning for Large Language Models [56.00992369295851]
  Open-sourced Large Language Models (LLMs) have achieved great success in various NLP tasks, however, they are still far inferior to API-based models when acting as agents. This paper delivers three key observations: (1) the current agent training corpus is entangled with both formats following and agent reasoning, which significantly shifts from the distribution of its pre-training data; (2) LLMs exhibit different learning speeds on the capabilities required by agent tasks; and (3) current approaches have side-effects when improving agent abilities by introducing hallucinations. We propose Agent-FLAN to effectively Fine-tune LANguage models for Agents.
  arXiv  Detail & Related papers  (2024-03-19T16:26:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.