Model-Free Reinforcement Learning for Asset Allocation
- URL: http://arxiv.org/abs/2209.10458v1
- Date: Wed, 21 Sep 2022 16:00:24 GMT
- Title: Model-Free Reinforcement Learning for Asset Allocation
- Authors: Adebayo Oshingbesan, Eniola Ajiboye, Peruth Kamashazi, Timothy Mbaka
- Abstract summary: This study investigated the performance of reinforcement learning when applied to portfolio management using model-free deep RL agents.
We trained several RL agents on real-world stock prices to learn how to perform asset allocation.
Four RL agents (A2C, SAC, PPO, and TRPO) outperformed the best baseline, MPT, overall.
- Score: 0.0
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Asset allocation (or portfolio management) is the task of determining how to
optimally allocate funds of a finite budget into a range of financial
instruments/assets such as stocks. This study investigated the performance of
reinforcement learning (RL) when applied to portfolio management using
model-free deep RL agents. We trained several RL agents on real-world stock
prices to learn how to perform asset allocation. We compared the performance of
these RL agents against some baseline agents. We also compared the RL agents
among themselves to understand which classes of agents performed better. From
our analysis, RL agents can perform the task of portfolio management since they
significantly outperformed two of the baseline agents (random allocation and
uniform allocation). Four RL agents (A2C, SAC, PPO, and TRPO) outperformed the
best baseline, MPT, overall. This shows the abilities of RL agents to uncover
more profitable trading strategies. Furthermore, there were no significant
performance differences between value-based and policy-based RL agents.
Actor-critic agents performed better than other types of agents. Also,
on-policy agents performed better than off-policy agents because they are
better at policy evaluation and sample efficiency is not a significant problem
in portfolio management. This study shows that RL agents can substantially
improve asset allocation since they outperform strong baselines. On-policy,
actor-critic RL agents showed the most promise based on our analysis.
Related papers
- ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RL [80.10358123795946]
We develop a framework for building multi-turn RL algorithms for fine-tuning large language models.
Our framework adopts a hierarchical RL approach and runs two RL algorithms in parallel.
Empirically, we find that ArCHer significantly improves efficiency and performance on agent tasks.
arXiv Detail & Related papers (2024-02-29T18:45:56Z) - Combining Transformer based Deep Reinforcement Learning with
Black-Litterman Model for Portfolio Optimization [0.0]
As a model-free algorithm, deep reinforcement learning (DRL) agent learns and makes decisions by interacting with the environment in an unsupervised way.
We propose a hybrid portfolio optimization model combining the DRL agent and the Black-Litterman (BL) model.
Our DRL agent significantly outperforms various comparison portfolio choice strategies and alternative DRL frameworks by at least 42% in terms of accumulated return.
arXiv Detail & Related papers (2024-02-23T16:01:37Z) - Learning From Good Trajectories in Offline Multi-Agent Reinforcement
Learning [98.07495732562654]
offline multi-agent reinforcement learning (MARL) aims to learn effective multi-agent policies from pre-collected datasets.
One agent learned by offline MARL often inherits this random policy, jeopardizing the performance of the entire team.
We propose a novel framework called Shared Individual Trajectories (SIT) to address this problem.
arXiv Detail & Related papers (2022-11-28T18:11:26Z) - Imitate then Transcend: Multi-Agent Optimal Execution with Dual-Window
Denoise PPO [13.05016423016994]
A novel framework for solving the optimal execution and placement problems using reinforcement learning (RL) with imitation was proposed.
The RL agents trained from the proposed framework consistently outperformed the industry benchmark time-weighted average price (TWAP) strategy in execution cost.
arXiv Detail & Related papers (2022-06-21T21:25:30Z) - Learning Cooperative Multi-Agent Policies with Partial Reward Decoupling [13.915157044948364]
One of the preeminent obstacles to scaling multi-agent reinforcement learning is assigning credit to individual agents' actions.
In this paper, we address this credit assignment problem with an approach that we call textitpartial reward decoupling (PRD)
PRD decomposes large cooperative multi-agent RL problems into decoupled subproblems involving subsets of agents, thereby simplifying credit assignment.
arXiv Detail & Related papers (2021-12-23T17:48:04Z) - Plan Better Amid Conservatism: Offline Multi-Agent Reinforcement
Learning with Actor Rectification [74.10976684469435]
offline reinforcement learning (RL) algorithms can be transferred to multi-agent settings directly.
We propose a simple yet effective method, Offline Multi-Agent RL with Actor Rectification (OMAR), to tackle this critical challenge.
OMAR significantly outperforms strong baselines with state-of-the-art performance in multi-agent continuous control benchmarks.
arXiv Detail & Related papers (2021-11-22T13:27:42Z) - Robust Reinforcement Learning on State Observations with Learned Optimal
Adversary [86.0846119254031]
We study the robustness of reinforcement learning with adversarially perturbed state observations.
With a fixed agent policy, we demonstrate that an optimal adversary to perturb state observations can be found.
For DRL settings, this leads to a novel empirical adversarial attack to RL agents via a learned adversary that is much stronger than previous ones.
arXiv Detail & Related papers (2021-01-21T05:38:52Z) - Robust Deep Reinforcement Learning against Adversarial Perturbations on
State Observations [88.94162416324505]
A deep reinforcement learning (DRL) agent observes its states through observations, which may contain natural measurement errors or adversarial noises.
Since the observations deviate from the true states, they can mislead the agent into making suboptimal actions.
We show that naively applying existing techniques on improving robustness for classification tasks, like adversarial training, is ineffective for many RL tasks.
arXiv Detail & Related papers (2020-03-19T17:59:59Z) - Scalable Multi-Agent Inverse Reinforcement Learning via
Actor-Attention-Critic [54.2180984002807]
Multi-agent adversarial inverse reinforcement learning (MA-AIRL) is a recent approach that applies single-agent AIRL to multi-agent problems.
We propose a multi-agent inverse RL algorithm that is more sample-efficient and scalable than previous works.
arXiv Detail & Related papers (2020-02-24T20:30:45Z) - A General Framework on Enhancing Portfolio Management with Reinforcement
Learning [3.6985496077087743]
Portfolio management concerns continuous reallocation of funds and assets across financial instruments to meet the desired returns to risk profile.
Deep reinforcement learning (RL) has gained increasing interest in portfolio management, where RL agents are trained base on financial data to optimize the asset reallocation process.
We propose a general RL framework for asset management that enables continuous asset weights, short selling and making decisions with relevant features.
arXiv Detail & Related papers (2019-11-26T23:41:06Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.