Market-based Architectures in RL and Beyond

• URL: http://arxiv.org/abs/2503.05828v1
• Date: Wed, 05 Mar 2025 19:09:29 GMT
• Title: Market-based Architectures in RL and Beyond
• Authors: Abhimanyu Pallavi Sudhir, Long Tran-Thanh,
• Abstract summary: We introduce a new type of market-based algorithm where the state itself is factored into several axes called goods''<n>We argue that market-based algorithms have the potential to address many current challenges in AI.
• Score: 15.819356579361845
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Market-based agents refer to reinforcement learning agents which determine their actions based on an internal market of sub-agents. We introduce a new type of market-based algorithm where the state itself is factored into several axes called ``goods'', which allows for greater specialization and parallelism than existing market-based RL algorithms. Furthermore, we argue that market-based algorithms have the potential to address many current challenges in AI, such as search, dynamic scaling and complete feedback, and demonstrate that they may be seen to generalize neural networks; finally, we list some novel ways that market algorithms may be applied in conjunction with Large Language Models for immediate practical applicability.

Related papers

• Accelerating Vehicle Routing via AI-Initialized Genetic Algorithms [55.78505925402658]
  Vehicle Routing Problems (VRP) are an extension of the Traveling Salesperson Problem and are a fundamental NP-hard challenge in Evolutionary optimization. We introduce a novel optimization framework that uses a reinforcement learning agent - trained on prior instances - to quickly generate initial solutions, which are then further optimized by genetic algorithms. For example, EARLI handles vehicle routing with 500 locations within 1s, 10x faster than current solvers for the same solution quality, enabling applications like real-time and interactive routing.
  arXiv  Detail & Related papers  (2025-04-08T15:21:01Z)
• By Fair Means or Foul: Quantifying Collusion in a Market Simulation with Deep Reinforcement Learning [1.5249435285717095]
  This research employs an experimental oligopoly model of repeated price competition. We investigate the strategies and emerging pricing patterns developed by the agents, which may lead to a collusive outcome. Our findings indicate that RL-based AI agents converge to a collusive state characterized by the charging of supracompetitive prices.
  arXiv  Detail & Related papers  (2024-06-04T15:35:08Z)
• A Network Simulation of OTC Markets with Multiple Agents [3.8944986367855963]
  We present a novel approach to simulating an over-the-counter (OTC) financial market in which trades are intermediated solely by market makers. We show that our network-based model can lend insights into the effect of market-structure on price-action.
  arXiv  Detail & Related papers  (2024-05-03T20:45:00Z)
• Developing A Multi-Agent and Self-Adaptive Framework with Deep Reinforcement Learning for Dynamic Portfolio Risk Management [1.2016264781280588]
  A multi-agent reinforcement learning (RL) approach is proposed to balance the trade-off between the overall portfolio returns and their potential risks. The obtained empirical results clearly reveal the potential strengths of our proposed MASA framework.
  arXiv  Detail & Related papers  (2024-02-01T11:31:26Z)
• Logic-Q: Improving Deep Reinforcement Learning-based Quantitative Trading via Program Sketch-based Tuning [9.039809980024852]
  We propose a universal logic-guided deep reinforcement learning framework for Q-trading, called Logic-Q. In particular, Logic-Q adopts the program synthesis by sketching paradigm and introduces a logic-guided model design that leverages a lightweight, plug-and-play market trend-aware program sketch to determine the market trend. Extensive evaluations of two popular quantitative trading tasks demonstrate that Logic-Q can significantly improve the performance of previous state-of-the-art DRL trading strategies.
  arXiv  Detail & Related papers  (2023-10-09T09:20:13Z)
• HireVAE: An Online and Adaptive Factor Model Based on Hierarchical and Regime-Switch VAE [113.47287249524008]
  It is still an open question to build a factor model that can conduct stock prediction in an online and adaptive setting. We propose the first deep learning based online and adaptive factor model, HireVAE, at the core of which is a hierarchical latent space that embeds the relationship between the market situation and stock-wise latent factors. Across four commonly used real stock market benchmarks, the proposed HireVAE demonstrate superior performance in terms of active returns over previous methods.
  arXiv  Detail & Related papers  (2023-06-05T12:58:13Z)
• Retrieval-Enhanced Machine Learning [110.5237983180089]
  We describe a generic retrieval-enhanced machine learning framework, which includes a number of existing models as special cases. REML challenges information retrieval conventions, presenting opportunities for novel advances in core areas, including optimization. REML research agenda lays a foundation for a new style of information access research and paves a path towards advancing machine learning and artificial intelligence.
  arXiv  Detail & Related papers  (2022-05-02T21:42:45Z)
• Bayesian Bilinear Neural Network for Predicting the Mid-price Dynamics in Limit-Order Book Markets [84.90242084523565]
  Traditional time-series econometric methods often appear incapable of capturing the true complexity of the multi-level interactions driving the price dynamics. By adopting a state-of-the-art second-order optimization algorithm, we train a Bayesian bilinear neural network with temporal attention. By addressing the use of predictive distributions to analyze errors and uncertainties associated with the estimated parameters and model forecasts, we thoroughly compare our Bayesian model with traditional ML alternatives.
  arXiv  Detail & Related papers  (2022-03-07T18:59:54Z)
• Deep Q-Learning Market Makers in a Multi-Agent Simulated Stock Market [58.720142291102135]
  This paper focuses precisely on the study of these markets makers strategies from an agent-based perspective. We propose the application of Reinforcement Learning (RL) for the creation of intelligent market markers in simulated stock markets.
  arXiv  Detail & Related papers  (2021-12-08T14:55:21Z)
• OSOUM Framework for Trading Data Research [79.0383470835073]
  We supply, to the best of our knowledge, the first open source simulation platform, Open SOUrce Market Simulator (OSOUM) to analyze trading markets and specifically data markets. We describe and implement a specific data market model, consisting of two types of agents: sellers who own various datasets available for acquisition, and buyers searching for relevant and beneficial datasets for purchase. Although commercial frameworks, intended for handling data markets, already exist, we provide a free and extensive end-to-end research tool for simulating possible behavior for both buyers and sellers participating in (data) markets.
  arXiv  Detail & Related papers  (2021-02-18T09:20:26Z)
• Reinforcement Learning as Iterative and Amortised Inference [62.997667081978825]
  We use the control as inference framework to outline a novel classification scheme based on amortised and iterative inference. We show that taking this perspective allows us to identify parts of the algorithmic design space which have been relatively unexplored.
  arXiv  Detail & Related papers  (2020-06-13T16:10:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.