End-to-End Policy Learning of a Statistical Arbitrage Autoencoder Architecture

• URL: http://arxiv.org/abs/2402.08233v1
• Date: Tue, 13 Feb 2024 05:53:00 GMT
• Title: End-to-End Policy Learning of a Statistical Arbitrage Autoencoder Architecture
• Abstract summary: We study the utility of Autoencoder architectures in Statistical Arbitrage (StatArb) We employ a standard Autoencoder trained on US stock returns to derive trading strategies based on the Ornstein-Uhlenbeck (OU) process. We take a policy-learning approach and embed the Autoencoder network into a neural network representation of a space of portfolio trading policies.
• Score: 0.5439020425818999
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In Statistical Arbitrage (StatArb), classical mean reversion trading strategies typically hinge on asset-pricing or PCA based models to identify the mean of a synthetic asset. Once such a (linear) model is identified, a separate mean reversion strategy is then devised to generate a trading signal. With a view of generalising such an approach and turning it truly data-driven, we study the utility of Autoencoder architectures in StatArb. As a first approach, we employ a standard Autoencoder trained on US stock returns to derive trading strategies based on the Ornstein-Uhlenbeck (OU) process. To further enhance this model, we take a policy-learning approach and embed the Autoencoder network into a neural network representation of a space of portfolio trading policies. This integration outputs portfolio allocations directly and is end-to-end trainable by backpropagation of the risk-adjusted returns of the neural policy. Our findings demonstrate that this innovative end-to-end policy learning approach not only simplifies the strategy development process, but also yields superior gross returns over its competitors illustrating the potential of end-to-end training over classical two-stage approaches.

Related papers

• Group-Relative REINFORCE Is Secretly an Off-Policy Algorithm: Demystifying Some Myths About GRPO and Its Friends [64.71326476563213]
  Off-policy reinforcement learning for large language models (LLMs) is attracting growing interest.<n>We present a first-principles derivation for grouprelative REINFORCE without assuming a specific training data distribution.<n>This perspective yields two general principles for adapting REINFORCE to off-policy settings.
  arXiv  Detail & Related papers  (2025-09-29T02:34:54Z)
• In-Context Learning Strategies Emerge Rationally [36.21854069812912]
  We show that when trained to learn a mixture of tasks, strategies learned by a model for performing ICL can be captured by a family of Bayesian predictors.<n>We develop a hierarchical Bayesian framework that almost perfectly predicts Transformer next-token predictions throughout training.<n>Our work advances an explanatory and predictive account of ICL grounded in tradeoffs between strategy loss and complexity.
  arXiv  Detail & Related papers  (2025-06-21T23:49:08Z)
• From Local Patterns to Global Understanding: Cross-Stock Trend Integration for Enhanced Predictive Modeling [9.531938681666471]
  We propose a novel method that merges local patterns into a global understanding through cross-stock pattern integration.<n>Our strategy is inspired by Federated Learning (FL), a paradigm designed for decentralized model training.<n>In our adaptation, we train models on individual stock data and iteratively merge them to create a unified global model.
  arXiv  Detail & Related papers  (2025-05-22T12:04:10Z)
• Your Offline Policy is Not Trustworthy: Bilevel Reinforcement Learning for Sequential Portfolio Optimization [82.03139922490796]
  Reinforcement learning (RL) has shown significant promise for sequential portfolio optimization tasks, such as stock trading, where the objective is to maximize cumulative returns while minimizing risks using historical data.<n>Traditional RL approaches often produce policies that merely memorize the optimal yet impractical buying and selling behaviors within the fixed dataset.<n>Our approach frames portfolio optimization as a new type of partial-offline RL problem and makes two technical contributions.
  arXiv  Detail & Related papers  (2025-05-19T06:37:25Z)
• Model Steering: Learning with a Reference Model Improves Generalization Bounds and Scaling Laws [52.10468229008941]
  This paper formalizes an emerging learning paradigm that uses a trained model as a reference to guide and enhance the training of a target model through strategic data selection or weighting.<n>We provide theoretical insights into why this approach improves generalization and data efficiency compared to training without a reference model.<n>Building on these insights, we introduce a novel method for Contrastive Language-Image Pretraining with a reference model, termed DRRho-CLIP.
  arXiv  Detail & Related papers  (2025-05-10T16:55:03Z)
• Dense Policy: Bidirectional Autoregressive Learning of Actions [51.60428100831717]
  This paper introduces a bidirectionally expanded learning approach, termed Dense Policy, to establish a new paradigm for autoregressive policies in action prediction. It employs a lightweight encoder-only architecture to iteratively unfold the action sequence from an initial single frame into the target sequence in a coarse-to-fine manner. Experiments validate that our dense policy has superior autoregressive learning capabilities and can surpass existing holistic generative policies.
  arXiv  Detail & Related papers  (2025-03-17T14:28:08Z)
• Complexity-Aware Deep Symbolic Regression with Robust Risk-Seeking Policy Gradients [20.941908494137806]
  We propose a novel deep symbolic regression approach to enhance the robustness and interpretability of data-driven mathematical expression discovery.<n>Our work is aligned with the popular DSR framework which focuses on learning a data-specific expression generator.
  arXiv  Detail & Related papers  (2024-06-10T19:29:10Z)
• Jointly Training and Pruning CNNs via Learnable Agent Guidance and Alignment [69.33930972652594]
  We propose a novel structural pruning approach to jointly learn the weights and structurally prune architectures of CNN models. The core element of our method is a Reinforcement Learning (RL) agent whose actions determine the pruning ratios of the CNN model's layers. We conduct the joint training and pruning by iteratively training the model's weights and the agent's policy.
  arXiv  Detail & Related papers  (2024-03-28T15:22:29Z)
• Deep Inventory Management [3.578617477295742]
  We present a Deep Reinforcement Learning approach to solving a periodic review inventory control system. We show that several policy learning approaches are competitive with or outperform classical baseline approaches.
  arXiv  Detail & Related papers  (2022-10-06T18:00:25Z)
• Strategic Decision-Making in the Presence of Information Asymmetry: Provably Efficient RL with Algorithmic Instruments [55.41685740015095]
  We study offline reinforcement learning under a novel model called strategic MDP. We propose a novel algorithm, Pessimistic policy Learning with Algorithmic iNstruments (PLAN)
  arXiv  Detail & Related papers  (2022-08-23T15:32:44Z)
• An intelligent algorithmic trading based on a risk-return reinforcement learning algorithm [0.0]
  This scientific paper propose a novel portfolio optimization model using an improved deep reinforcement learning algorithm. The proposed algorithm is based on actor-critic architecture, in which the main task of critical network is to learn the distribution of portfolio cumulative return. A multi-process method is used, called Ape-x, to accelerate the speed of deep reinforcement learning training.
  arXiv  Detail & Related papers  (2022-08-23T03:20:06Z)
• Backward Imitation and Forward Reinforcement Learning via Bi-directional Model Rollouts [11.4219428942199]
  Traditional model-based reinforcement learning (RL) methods generate forward rollout traces using the learnt dynamics model. In this paper, we propose the backward imitation and forward reinforcement learning (BIFRL) framework. BIFRL empowers the agent to both reach to and explore from high-value states in a more efficient manner.
  arXiv  Detail & Related papers  (2022-08-04T04:04:05Z)
• Machine learning method for return direction forecasting of Exchange Traded Funds using classification and regression models [0.0]
  This article aims to propose and apply a machine learning method to analyze the direction of returns from Exchange Traded Funds (ETFs) Regression and classification models were applied, using standard datasets from Brazilian and American markets. In terms of risk and return, the models mostly performed better than the control metrics.
  arXiv  Detail & Related papers  (2022-05-25T12:54:46Z)
• Federated Learning Aggregation: New Robust Algorithms with Guarantees [63.96013144017572]
  Federated learning has been recently proposed for distributed model training at the edge. This paper presents a complete general mathematical convergence analysis to evaluate aggregation strategies in a federated learning framework. We derive novel aggregation algorithms which are able to modify their model architecture by differentiating client contributions according to the value of their losses.
  arXiv  Detail & Related papers  (2022-05-22T16:37:53Z)
• Minimax Model Learning [42.65032356835701]
  We present a novel off-policy loss function for learning a transition model in model-based reinforcement learning. Our loss is derived from the off-policy policy evaluation objective with an emphasis on correcting distribution shift.
  arXiv  Detail & Related papers  (2021-03-02T23:16:36Z)
• COMBO: Conservative Offline Model-Based Policy Optimization [120.55713363569845]
  Uncertainty estimation with complex models, such as deep neural networks, can be difficult and unreliable. We develop a new model-based offline RL algorithm, COMBO, that regularizes the value function on out-of-support state-actions. We find that COMBO consistently performs as well or better as compared to prior offline model-free and model-based methods.
  arXiv  Detail & Related papers  (2021-02-16T18:50:32Z)
• Model-Based Machine Learning for Communications [110.47840878388453]
  We review existing strategies for combining model-based algorithms and machine learning from a high level perspective. We focus on symbol detection, which is one of the fundamental tasks of communication receivers.
  arXiv  Detail & Related papers  (2021-01-12T19:55:34Z)
• MOPO: Model-based Offline Policy Optimization [183.6449600580806]
  offline reinforcement learning (RL) refers to the problem of learning policies entirely from a large batch of previously collected data. We show that an existing model-based RL algorithm already produces significant gains in the offline setting. We propose to modify the existing model-based RL methods by applying them with rewards artificially penalized by the uncertainty of the dynamics.
  arXiv  Detail & Related papers  (2020-05-27T08:46:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.