AlphaEval: A Comprehensive and Efficient Evaluation Framework for Formula Alpha Mining

• URL: http://arxiv.org/abs/2508.13174v1
• Date: Sun, 10 Aug 2025 11:19:24 GMT
• Title: AlphaEval: A Comprehensive and Efficient Evaluation Framework for Formula Alpha Mining
• Authors: Hongjun Ding, Binqi Chen, Jinsheng Huang, Taian Guo, Zhengyang Mao, Guoyi Shao, Lutong Zou, Luchen Liu, Ming Zhang,
• Abstract summary: Formula alpha mining, which generates predictive signals from financial data, is critical for quantitative investment.<n>Existing evaluation metrics predominantly include backtesting and correlation-based measures.<n>We propose AlphaEval, a unified, parallelizable, and backtest-free evaluation framework for automated alpha mining models.
• Score: 6.167227740097627
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Formula alpha mining, which generates predictive signals from financial data, is critical for quantitative investment. Although various algorithmic approaches-such as genetic programming, reinforcement learning, and large language models-have significantly expanded the capacity for alpha discovery, systematic evaluation remains a key challenge. Existing evaluation metrics predominantly include backtesting and correlation-based measures. Backtesting is computationally intensive, inherently sequential, and sensitive to specific strategy parameters. Correlation-based metrics, though efficient, assess only predictive ability and overlook other crucial properties such as temporal stability, robustness, diversity, and interpretability. Additionally, the closed-source nature of most existing alpha mining models hinders reproducibility and slows progress in this field. To address these issues, we propose AlphaEval, a unified, parallelizable, and backtest-free evaluation framework for automated alpha mining models. AlphaEval assesses the overall quality of generated alphas along five complementary dimensions: predictive power, stability, robustness to market perturbations, financial logic, and diversity. Extensive experiments across representative alpha mining algorithms demonstrate that AlphaEval achieves evaluation consistency comparable to comprehensive backtesting, while providing more comprehensive insights and higher efficiency. Furthermore, AlphaEval effectively identifies superior alphas compared to traditional single-metric screening approaches. All implementations and evaluation tools are open-sourced to promote reproducibility and community engagement.

Related papers

• Alpha-R1: Alpha Screening with LLM Reasoning via Reinforcement Learning [28.326583684637853]
  Signal decay and regime shifts pose recurring challenges for data-driven investment strategies in non-stationary markets.<n>Existing factor-based methods typically reduce alphas to numerical time series, overlooking the semantic rationale that determines when a factor is economically relevant.<n>We propose Alpha-R1, an 8B- parameter reasoning model trained via reinforcement learning for context-aware alpha screening.
  arXiv  Detail & Related papers  (2025-12-29T14:50:23Z)
• InnoGym: Benchmarking the Innovation Potential of AI Agents [74.64144272881414]
  InnoGym is the first benchmark designed to evaluate the innovation potential of AI agents.<n>InnoGym introduces two complementary metrics: performance gain, which measures improvement over the best-known solutions, and novelty, which captures methodological differences from prior approaches.
  arXiv  Detail & Related papers  (2025-12-01T16:03:04Z)
• Cognitive Alpha Mining via LLM-Driven Code-Based Evolution [29.71597480304934]
  We introduce the Cognitive Alpha Mining Framework (CogAlpha), which combines code-level alpha representation with LLM-driven reasoning and evolutionary search.<n>Treating LLMs as adaptive cognitive agents, our framework iteratively refines, mutates, and recombines alpha candidates through prompts and financial feedback.<n>Experiments on A-share equities demonstrate that CogAlpha consistently discovers alphas with superior predictive accuracy, robustness, and generalization over existing methods.
  arXiv  Detail & Related papers  (2025-11-24T07:45:59Z)
• Navigating the Alpha Jungle: An LLM-Powered MCTS Framework for Formulaic Factor Mining [8.53606484300001]
  This paper introduces a novel framework that integrates Large Language Models (LLMs) with Monte Carlo Tree Search (MCTS)<n>A key innovation is the guidance of MCTS exploration by rich, quantitative feedback from financial backtesting of each candidate factor.<n> Experimental results on real-world stock market data demonstrate that our LLM-based framework outperforms existing methods by mining alphas with superior predictive accuracy and trading performance.
  arXiv  Detail & Related papers  (2025-05-16T11:14:17Z)
• AlphaAgent: LLM-Driven Alpha Mining with Regularized Exploration to Counteract Alpha Decay [43.50447460231601]
  We propose AlphaAgent, an autonomous framework that integrates Large Language Models with ad hoc regularizations for mining decay-resistant alpha factors.<n>AlphaAgent consistently delivers significant alpha in Chinese CSI 500 and US S&P 500 markets over the past four years.<n> Notably, AlphaAgent showcases remarkable resistance to alpha decay, elevating the potential for yielding powerful factors.
  arXiv  Detail & Related papers  (2025-02-24T02:56:46Z)
• Reinforcement Learning under Latent Dynamics: Toward Statistical and Algorithmic Modularity [51.40558987254471]
  Real-world applications of reinforcement learning often involve environments where agents operate on complex, high-dimensional observations. This paper addresses the question of reinforcement learning under $textitgeneral$ latent dynamics from a statistical and algorithmic perspective.
  arXiv  Detail & Related papers  (2024-10-23T14:22:49Z)
• AlphaForge: A Framework to Mine and Dynamically Combine Formulaic Alpha Factors [14.80394452270726]
  This paper proposes a two-stage alpha generating framework AlphaForge, for alpha factor mining and factor combination.<n> Experiments conducted on real-world datasets demonstrate that our proposed model outperforms contemporary benchmarks in formulaic alpha factor mining.
  arXiv  Detail & Related papers  (2024-06-26T14:34:37Z)
• Position: AI Evaluation Should Learn from How We Test Humans [65.36614996495983]
  We argue that psychometrics, a theory originating in the 20th century for human assessment, could be a powerful solution to the challenges in today's AI evaluations.
  arXiv  Detail & Related papers  (2023-06-18T09:54:33Z)
• On the Robustness of Aspect-based Sentiment Analysis: Rethinking Model, Data, and Training [109.9218185711916]
  Aspect-based sentiment analysis (ABSA) aims at automatically inferring the specific sentiment polarities toward certain aspects of products or services behind social media texts or reviews. We propose to enhance the ABSA robustness by systematically rethinking the bottlenecks from all possible angles, including model, data, and training.
  arXiv  Detail & Related papers  (2023-04-19T11:07:43Z)
• Uncertainty Quantification for Deep Context-Aware Mobile Activity Recognition and Unknown Context Discovery [85.36948722680822]
  We develop a context-aware mixture of deep models termed the alpha-beta network. We improve accuracy and F score by 10% by identifying high-level contexts. In order to ensure training stability, we have used a clustering-based pre-training in both public and in-house datasets.
  arXiv  Detail & Related papers  (2020-03-03T19:35:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.