Reinforcement Learning Applied to Trading Systems: A Survey

• URL: http://arxiv.org/abs/2212.06064v1
• Date: Tue, 1 Nov 2022 21:26:12 GMT
• Title: Reinforcement Learning Applied to Trading Systems: A Survey
• Authors: Leonardo Kanashiro Felizardo, Francisco Caio Lima Paiva, Anna Helena Reali Costa, Emilio Del-Moral-Hernandez
• Abstract summary: The recent achievements and the notoriety of Reinforcement Learning have increased its adoption in trading tasks. This review attempts to promote the development of this field of study by researchers' commitment to standards adherence.
• Score: 5.118560450410779
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Financial domain tasks, such as trading in market exchanges, are challenging and have long attracted researchers. The recent achievements and the consequent notoriety of Reinforcement Learning (RL) have also increased its adoption in trading tasks. RL uses a framework with well-established formal concepts, which raises its attractiveness in learning profitable trading strategies. However, RL use without due attention in the financial area can prevent new researchers from following standards or failing to adopt relevant conceptual guidelines. In this work, we embrace the seminal RL technical fundamentals, concepts, and recommendations to perform a unified, theoretically-grounded examination and comparison of previous research that could serve as a structuring guide for the field of study. A selection of twenty-nine articles was reviewed under our classification that considers RL's most common formulations and design patterns from a large volume of available studies. This classification allowed for precise inspection of the most relevant aspects regarding data input, preprocessing, state and action composition, adopted RL techniques, evaluation setups, and overall results. Our analysis approach organized around fundamental RL concepts allowed for a clear identification of current system design best practices, gaps that require further investigation, and promising research opportunities. Finally, this review attempts to promote the development of this field of study by facilitating researchers' commitment to standards adherence and helping them to avoid straying away from the RL constructs' firm ground.

Related papers

• A Comprehensive Survey of Datasets, Theories, Variants, and Applications in Direct Preference Optimization [52.42860559005861]
  Direct Preference Optimization (DPO) has emerged as a promising approach for alignment. Despite DPO's various advancements and inherent limitations, an in-depth review of these aspects is currently lacking in the literature.
  arXiv  Detail & Related papers  (2024-10-21T02:27:24Z)
• Advances in Preference-based Reinforcement Learning: A Review [1.474723404975345]
  Preference-based reinforcement learning (PbRL) addresses that by utilizing human preferences as feedback from the experts instead of numeric rewards. We present a unified PbRL framework to include the newly emerging approaches that improve the scalability and efficiency of PbRL.
  arXiv  Detail & Related papers  (2024-08-21T18:57:12Z)
• Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methods [18.771658054884693]
  Large language models (LLMs) emerge as a promising avenue to augment reinforcement learning (RL) in aspects such as multi-task learning, sample efficiency, and high-level task planning. We propose a structured taxonomy to systematically categorize LLMs' functionalities in RL, including four roles: information processor, reward designer, decision-maker, and generator.
  arXiv  Detail & Related papers  (2024-03-30T08:28:08Z)
• Reinforcement Learning-assisted Evolutionary Algorithm: A Survey and Research Opportunities [63.258517066104446]
  Reinforcement learning integrated as a component in the evolutionary algorithm has demonstrated superior performance in recent years. We discuss the RL-EA integration method, the RL-assisted strategy adopted by RL-EA, and its applications according to the existing literature. In the applications of RL-EA section, we also demonstrate the excellent performance of RL-EA on several benchmarks and a range of public datasets.
  arXiv  Detail & Related papers  (2023-08-25T15:06:05Z)
• Provable Reward-Agnostic Preference-Based Reinforcement Learning [61.39541986848391]
  Preference-based Reinforcement Learning (PbRL) is a paradigm in which an RL agent learns to optimize a task using pair-wise preference-based feedback over trajectories. We propose a theoretical reward-agnostic PbRL framework where exploratory trajectories that enable accurate learning of hidden reward functions are acquired.
  arXiv  Detail & Related papers  (2023-05-29T15:00:09Z)
• Evolutionary Reinforcement Learning: A Survey [31.112066295496003]
  Reinforcement learning (RL) is a machine learning approach that trains agents to maximize cumulative rewards through interactions with environments. This article presents a comprehensive survey of state-of-the-art methods for integrating EC into RL, referred to as evolutionary reinforcement learning (EvoRL)
  arXiv  Detail & Related papers  (2023-03-07T01:38:42Z)
• Ensemble Reinforcement Learning: A Survey [43.17635633600716]
  Reinforcement Learning (RL) has emerged as a highly effective technique for addressing various scientific and applied problems. In response, ensemble reinforcement learning (ERL), a promising approach that combines the benefits of both RL and ensemble learning (EL), has gained widespread popularity. ERL leverages multiple models or training algorithms to comprehensively explore the problem space and possesses strong generalization capabilities.
  arXiv  Detail & Related papers  (2023-03-05T09:26:44Z)
• A Survey of Meta-Reinforcement Learning [69.76165430793571]
  We cast the development of better RL algorithms as a machine learning problem itself in a process called meta-RL. We discuss how, at a high level, meta-RL research can be clustered based on the presence of a task distribution and the learning budget available for each individual task. We conclude by presenting the open problems on the path to making meta-RL part of the standard toolbox for a deep RL practitioner.
  arXiv  Detail & Related papers  (2023-01-19T12:01:41Z)
• A Comprehensive Survey of Data Augmentation in Visual Reinforcement Learning [53.35317176453194]
  Data augmentation (DA) has become a widely used technique in visual RL for acquiring sample-efficient and generalizable policies. We present a principled taxonomy of the existing augmentation techniques used in visual RL and conduct an in-depth discussion on how to better leverage augmented data. As the first comprehensive survey of DA in visual RL, this work is expected to offer valuable guidance to this emerging field.
  arXiv  Detail & Related papers  (2022-10-10T11:01:57Z)
• Survey on Fair Reinforcement Learning: Theory and Practice [9.783469272270896]
  We provide an extensive overview of fairness approaches that have been implemented via a reinforcement learning (RL) framework. We discuss various practical applications in which RL methods have been applied to achieve a fair solution with high accuracy. We highlight a few major issues to explore in order to advance the field of fair-RL.
  arXiv  Detail & Related papers  (2022-05-20T09:07:28Z)
• Variational Empowerment as Representation Learning for Goal-Based Reinforcement Learning [114.07623388322048]
  We discuss how the standard goal-conditioned RL (GCRL) is encapsulated by the objective variational empowerment. Our work lays a novel foundation from which to evaluate, analyze, and develop representation learning techniques in goal-based RL.
  arXiv  Detail & Related papers  (2021-06-02T18:12:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.