Commodities Trading through Deep Policy Gradient Methods
- URL: http://arxiv.org/abs/2309.00630v1
- Date: Thu, 10 Aug 2023 17:21:12 GMT
- Title: Commodities Trading through Deep Policy Gradient Methods
- Authors: Jonas Hanetho
- Abstract summary: It formulates the commodities trading problem as a continuous, discrete-time dynamical system.
Two policy algorithms, namely actor-based and actor-critic-based approaches, are introduced.
Backtesting on front-month natural gas futures demonstrates that DRL models increase the Sharpe ratio by $83%$ compared to the buy-and-hold baseline.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Algorithmic trading has gained attention due to its potential for generating
superior returns. This paper investigates the effectiveness of deep
reinforcement learning (DRL) methods in algorithmic commodities trading. It
formulates the commodities trading problem as a continuous, discrete-time
stochastic dynamical system. The proposed system employs a novel
time-discretization scheme that adapts to market volatility, enhancing the
statistical properties of subsampled financial time series. To optimize
transaction-cost- and risk-sensitive trading agents, two policy gradient
algorithms, namely actor-based and actor-critic-based approaches, are
introduced. These agents utilize CNNs and LSTMs as parametric function
approximators to map historical price observations to market
positions.Backtesting on front-month natural gas futures demonstrates that DRL
models increase the Sharpe ratio by $83\%$ compared to the buy-and-hold
baseline. Additionally, the risk profile of the agents can be customized
through a hyperparameter that regulates risk sensitivity in the reward function
during the optimization process. The actor-based models outperform the
actor-critic-based models, while the CNN-based models show a slight performance
advantage over the LSTM-based models.
Related papers
- Scaling Laws for Reward Model Overoptimization in Direct Alignment Algorithms [50.808123629394245]
Reinforcement Learning from Human Feedback (RLHF) has been crucial to the recent success of Large Language Models (LLMs)
This work formulates and formalizes the reward over-optimization or hacking problem for DAAs.
We find that DAA methods deteriorate not only across a wide range of KL budgets but also often before even a single epoch of the dataset is completed.
arXiv Detail & Related papers (2024-06-05T03:41:37Z) - RVRAE: A Dynamic Factor Model Based on Variational Recurrent Autoencoder
for Stock Returns Prediction [5.281288833470249]
RVRAE is a probabilistic approach that addresses the temporal dependencies and noise in market data.
It is adept at risk modeling in volatile stock markets, estimating variances from latent space distributions while also predicting returns.
arXiv Detail & Related papers (2024-03-04T21:48:32Z) - Deep Policy Gradient Methods in Commodity Markets [0.0]
Traders play an important role in stabilizing markets by providing liquidity and reducing volatility.
This thesis investigates the effectiveness of deep reinforcement learning methods in commodities trading.
arXiv Detail & Related papers (2023-06-14T11:50:23Z) - Structured Dynamic Pricing: Optimal Regret in a Global Shrinkage Model [50.06663781566795]
We consider a dynamic model with the consumers' preferences as well as price sensitivity varying over time.
We measure the performance of a dynamic pricing policy via regret, which is the expected revenue loss compared to a clairvoyant that knows the sequence of model parameters in advance.
Our regret analysis results not only demonstrate optimality of the proposed policy but also show that for policy planning it is essential to incorporate available structural information.
arXiv Detail & Related papers (2023-03-28T00:23:23Z) - DeepVol: Volatility Forecasting from High-Frequency Data with Dilated
Causal Convolutions [78.6363825307044]
We propose DeepVol, a model based on Dilated Causal Convolutions to forecast day-ahead volatility by using high-frequency data.
We show that the dilated convolutional filters are ideally suited to extract relevant information from intraday financial data.
arXiv Detail & Related papers (2022-09-23T16:13:47Z) - Machine learning method for return direction forecasting of Exchange
Traded Funds using classification and regression models [0.0]
This article aims to propose and apply a machine learning method to analyze the direction of returns from Exchange Traded Funds (ETFs)
Regression and classification models were applied, using standard datasets from Brazilian and American markets.
In terms of risk and return, the models mostly performed better than the control metrics.
arXiv Detail & Related papers (2022-05-25T12:54:46Z) - Bayesian Bilinear Neural Network for Predicting the Mid-price Dynamics
in Limit-Order Book Markets [84.90242084523565]
Traditional time-series econometric methods often appear incapable of capturing the true complexity of the multi-level interactions driving the price dynamics.
By adopting a state-of-the-art second-order optimization algorithm, we train a Bayesian bilinear neural network with temporal attention.
By addressing the use of predictive distributions to analyze errors and uncertainties associated with the estimated parameters and model forecasts, we thoroughly compare our Bayesian model with traditional ML alternatives.
arXiv Detail & Related papers (2022-03-07T18:59:54Z) - Bitcoin Transaction Strategy Construction Based on Deep Reinforcement
Learning [8.431365407963629]
This study proposes a framework for automatic high-frequency bitcoin transactions based on a deep reinforcement learning algorithm-proximal policy optimization (PPO)
The proposed framework can earn excess returns through both the period of volatility and surge, which opens the door to research on building a single cryptocurrency trading strategy based on deep learning.
arXiv Detail & Related papers (2021-09-30T01:24:03Z) - GA-MSSR: Genetic Algorithm Maximizing Sharpe and Sterling Ratio Method
for RoboTrading [0.4568777157687961]
Foreign exchange is the largest financial market in the world.
Most literature used historical price information and technical indicators for training.
To address this problem, we designed trading rule features that are derived from technical indicators and trading rules.
arXiv Detail & Related papers (2020-08-16T05:33:35Z) - MOPO: Model-based Offline Policy Optimization [183.6449600580806]
offline reinforcement learning (RL) refers to the problem of learning policies entirely from a large batch of previously collected data.
We show that an existing model-based RL algorithm already produces significant gains in the offline setting.
We propose to modify the existing model-based RL methods by applying them with rewards artificially penalized by the uncertainty of the dynamics.
arXiv Detail & Related papers (2020-05-27T08:46:41Z) - Adaptive Control and Regret Minimization in Linear Quadratic Gaussian
(LQG) Setting [91.43582419264763]
We propose LqgOpt, a novel reinforcement learning algorithm based on the principle of optimism in the face of uncertainty.
LqgOpt efficiently explores the system dynamics, estimates the model parameters up to their confidence interval, and deploys the controller of the most optimistic model.
arXiv Detail & Related papers (2020-03-12T19:56:38Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.