Temporal Leakage in Financial News NLP: A Multi-Architecture Audit with a Regime-Specific M&A Signal

• URL: http://arxiv.org/abs/2608.17223v1
• Date: Tue, 18 Aug 2026 00:23:25 GMT
• Title: Temporal Leakage in Financial News NLP: A Multi-Architecture Audit with a Regime-Specific M&A Signal
• Abstract summary: We audit dependence on a 49,799-article corpus across 16 feature-model combinations spanning TF-IDF, MiniLM, FinBERT, and fine-tuned RoBERTa-large / DeBERTa-v3-large.<n>Random splits inflate MCC by $1.1times$ to $6.5times$, tracking model capacity and feature richness, and end-to-end FinBERT fine-tuning re-amplifies rather than closes the gap.
• Score: 13.5023809267948
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Financial-news direction prediction has become a popular NLP benchmark, yet reported gains depend critically on whether the train-test split is chronological or random, i.e., on temporal leakage. We audit this dependence on a 49,799-article corpus across 16 feature-model combinations spanning TF-IDF, MiniLM, FinBERT, and fine-tuned RoBERTa-large / DeBERTa-v3-large, plus separate zero/few-shot and LoRA probes of Llama-3 and Qwen2.5 LLMs: random splits inflate MCC by $1.1\times$ to $6.5\times$, tracking model capacity and feature richness, and end-to-end FinBERT fine-tuning re-amplifies rather than closes the gap (size-matched ratio $1.75\times$). Conditioning on event type, mergers and acquisitions (M&A) is the only audited category with a positive locked-test signal under near-temporal chronological evaluation (TF-IDF MCC $= 0.138$ train-only, $0.068$ under train$\cup$val refit; 10,000-permutation $p < 10^{-3}$); the signal does not transfer to FNSPID's 2009-2020 U.S. corpus, localising the headline to our 2024-2025 European-tilted M&A semantics rather than a universal predictor. Three independent role labellers converge on acquirer-tagged articles as the signal locus, a power-limited qualitative convergence rather than a hypothesis-tested asymmetry. Chronological splitting plays for financial NLP the role characteristics-purging plays for asset pricing: it strips the predictable, stale component of news and leaves a residual that is small, event-localized, and lexically shallow. We advocate leakage audits as a required disclosure for financial-NLP benchmarks.

Related papers

• SANE: State Anomaly Neutralization for Stable Extreme-Context Delta-Rule Models [53.86918766240095]
  Delta-Rule recurrent models maintain a fixed-size state, enabling $O(1)$ inference memory but potentially becoming unstable under extreme-context extrapolation.<n>We propose textbfState Anomaly Neutralization (SANE), which applies adaptive $tanh$ compression at chunk boundaries while preserving the intra-chunk parallel structure.
  arXiv  Detail & Related papers  (2026-08-23T10:41:07Z)
• RA-FinBERT: Rule-aware LoRA adaptation for low-resource financial sentiment classification [8.831449509561043]
  Rule-Aware FinBERT (RA-FinBERT) is a framework that integrates low-rank adaptation (LoRA) with three continuous VADER-derived sentiment proportions.<n> RA-FinBERT achieved 69.89% accuracy and a macro F1 score of 0.634, compared with 63.44% and 0.526 for text-only FinBERT.
  arXiv  Detail & Related papers  (2026-08-10T16:52:19Z)
• From Financial Sentiment Classification to Return Predictability: A QLoRA Benchmark of Large Language Models [0.0]
  We construct a unified three-class benchmark from five financial text datasets.<n>We compare TF--IDF Naive Bayes, off-the-shelf FinBERT and Financial-RoBERTa encoders.<n>We evaluate economic validity on a temporally separate 2019 Benzinga sample.
  arXiv  Detail & Related papers  (2026-08-04T19:57:40Z)
• Beyond Sentiment: Structured Information Extraction from Financial News [5.502693922654938]
  FinBERT sentiment features exhibit strong predictive power under nonlinear models.<n>We find that compressing news into a single sentiment score incurs substantial information loss.
  arXiv  Detail & Related papers  (2026-07-30T16:41:55Z)
• One Human, $N$ Agents: Audit-Budget Allocation for LLM Agent Fleets under Miscalibrated, Correlated Confidence [41.99844472131922]
  A single human must audit $N$ LLM agents under a budget of $B ll N$ audits per round.<n>We model this as budgeted noisy inspection over a two-level Gaussian copula and locate the miscalibration threshold $*$ past which confidence-ranked auditing is emphworse than random.
  arXiv  Detail & Related papers  (2026-07-30T14:52:36Z)
• Scaling Laws for Agent Harnesses via Effective Feedback Compute [53.68149869349268]
  emphEffective Feedback Compute (EFC) is a trace-level scaling coordinate that credits feedback only when it is informative, valid, non-redundant, and retained for subsequent decisions.<n>EFC-based coordinates consistently predict failure rates better than raw-compute baselines.
  arXiv  Detail & Related papers  (2026-05-28T09:45:47Z)
• FinHarness: An Inline Lifecycle Safety Harness for Finance LLM Agents [45.708389696454304]
  FinHarness wraps a finance agent end-to-end with three components.<n>Fired risk factors are re-injected into the agent input as ex-ante evidence, enabling the agent to refuse, re-plan, or approve on its own.
  arXiv  Detail & Related papers  (2026-05-26T17:41:01Z)
• Semantic State Abstraction Interfaces for LLM-Augmented Portfolio Decisions: Multi-Axis News Decomposition and RL Diagnostics [0.0]
  We introduce Semantic State Abstraction Interfaces (SSAI), a methodological template for mapping sparse unstructured text into $K$ auditable.<n>Our contribution is the framework and its evaluation protocol, not a claim that SSAI outperforms denser alternatives.<n>We position SSAI as an interpretability-performance diagnostic and reusable protocol for sparse-text decision systems.
  arXiv  Detail & Related papers  (2026-05-07T11:37:40Z)
• FinSTaR: Towards Financial Reasoning with Time Series Reasoning Models [12.754011132549167]
  Time series (TSRMs) have shown promising capabilities in general domains, yet they consistently fail on financial domain.<n>We propose a general 2x2 capability taxonomy for TSRMs by crossing 1) single-entity vs. multi-entity analysis with 2) assessment of the current state vs. prediction of future behavior.
  arXiv  Detail & Related papers  (2026-05-05T07:46:39Z)
• Information Fidelity in Tool-Using LLM Agents: A Martingale Analysis of the Model Context Protocol [69.11739400975445]
  We introduce the first theoretical framework for analyzing error accumulation in Model Context Protocol (MCP) agents.<n>We show that cumulative distortion exhibits linear growth and high-probability deviations bounded by $O(sqrtT)$.<n>Key findings include: semantic weighting reduces distortion by 80%, and periodic re-grounding approximately every 9 steps suffices for error control.
  arXiv  Detail & Related papers  (2026-02-10T21:08:53Z)
• Almost Asymptotically Optimal Active Clustering Through Pairwise Observations [59.20614082241528]
  We propose a new analysis framework for clustering $M$ items into an unknown number of $K$ distinct groups using noisy and actively collected responses.<n>We establish a fundamental lower bound on the expected number of queries needed to achieve a desired confidence in the accuracy of the clustering.<n>We develop a computationally feasible variant of the Generalized Likelihood Ratio statistic and show that its performance gap to the lower bound can be accurately empirically estimated.
  arXiv  Detail & Related papers  (2026-02-05T14:16:47Z)
• Mind the Gap: A Causal Perspective on Bias Amplification in Prediction & Decision-Making [58.06306331390586]
  We introduce the notion of a margin complement, which measures how much a prediction score $S$ changes due to a thresholding operation. We show that under suitable causal assumptions, the influences of $X$ on the prediction score $S$ are equal to the influences of $X$ on the true outcome $Y$.
  arXiv  Detail & Related papers  (2024-05-24T11:22:19Z)
• Efficient and Sharp Off-Policy Evaluation in Robust Markov Decision Processes [44.974100402600165]
  We study the evaluation of a policy best-parametric and worst-case perturbations to a decision process (MDP) We use transition observations from the original MDP, whether they are generated under the same or a different policy. Our estimator is also estimated statistical inference using Wald confidence intervals.
  arXiv  Detail & Related papers  (2024-03-29T18:11:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.