Q-learning with online random forests

• URL: http://arxiv.org/abs/2204.03771v1
• Date: Thu, 7 Apr 2022 23:00:39 GMT
• Title: Q-learning with online random forests
• Authors: Joosung Min and Lloyd T. Elliott
• Abstract summary: We provide online random forests as $Q$-function approximators and propose a novel method wherein the random forest is grown as learning proceeds. We show that expanding forests improve performance, suggesting that expanding forests are viable for other applications of online random forests beyond the reinforcement learning setting.
• Score: 0.913755431537592
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: $Q$-learning is the most fundamental model-free reinforcement learning algorithm. Deployment of $Q$-learning requires approximation of the state-action value function (also known as the $Q$-function). In this work, we provide online random forests as $Q$-function approximators and propose a novel method wherein the random forest is grown as learning proceeds (through expanding forests). We demonstrate improved performance of our methods over state-of-the-art Deep $Q$-Networks in two OpenAI gyms (`blackjack' and `inverted pendulum') but not in the `lunar lander' gym. We suspect that the resilience to overfitting enjoyed by random forests recommends our method for common tasks that do not require a strong representation of the problem domain. We show that expanding forests (in which the number of trees increases as data comes in) improve performance, suggesting that expanding forests are viable for other applications of online random forests beyond the reinforcement learning setting.

Related papers

• TreeLoRA: Efficient Continual Learning via Layer-Wise LoRAs Guided by a Hierarchical Gradient-Similarity Tree [52.44403214958304]
  In this paper, we introduce TreeLoRA, a novel approach that constructs layer-wise adapters by leveraging hierarchical gradient similarity.<n>To reduce the computational burden of task similarity estimation, we employ bandit techniques to develop an algorithm based on lower confidence bounds.<n> experiments on both vision transformers (ViTs) and large language models (LLMs) demonstrate the effectiveness and efficiency of our approach.
  arXiv  Detail & Related papers  (2025-06-12T05:25:35Z)
• On the Learning with Augmented Class via Forests [17.606415934443554]
  We focus on learning with augmented class via forests, where an augmented class may appear in testing data yet not in training data.<n>We develop the Learning with Augmented Class via Forests approach, which constructs shallow forests according to the augmented Gini impurity.<n>We also develop deep neural forests via an optimization objective based on our augmented Gini impurity.
  arXiv  Detail & Related papers  (2025-05-14T11:22:22Z)
• DynFrs: An Efficient Framework for Machine Unlearning in Random Forest [2.315324942451179]
  DynFrs is a framework designed to enable efficient machine unlearning in Random Forests. In experiments, applying Dynfrs on Extremely Trees yields substantial improvements.
  arXiv  Detail & Related papers  (2024-10-02T14:20:30Z)
• LiteSearch: Efficacious Tree Search for LLM [70.29796112457662]
  This study introduces a novel guided tree search algorithm with dynamic node selection and node-level exploration budget. Experiments conducted on the GSM8K and TabMWP datasets demonstrate that our approach enjoys significantly lower computational costs compared to baseline methods.
  arXiv  Detail & Related papers  (2024-06-29T05:14:04Z)
• Adaptive Split Balancing for Optimal Random Forest [8.916614661563893]
  We propose a new random forest algorithm that constructs the trees using a novel adaptive split-balancing method. Our method achieves optimality in simple, smooth scenarios while adaptively learning the tree structure from the data.
  arXiv  Detail & Related papers  (2024-02-17T09:10:40Z)
• ForensicsForest Family: A Series of Multi-scale Hierarchical Cascade Forests for Detecting GAN-generated Faces [53.739014757621376]
  We describe a simple and effective forest-based method set called em ForensicsForest Family to detect GAN-generate faces. ForenscisForest is a newly proposed Multi-scale Hierarchical Cascade Forest. Hybrid ForensicsForest integrates the CNN layers into models. Divide-and-Conquer ForensicsForest can construct a forest model using only a portion of training samplings.
  arXiv  Detail & Related papers  (2023-08-02T06:41:19Z)
• Neuroevolution-based Classifiers for Deforestation Detection in Tropical Forests [62.997667081978825]
  Millions of hectares of tropical forests are lost every year due to deforestation or degradation. Monitoring and deforestation detection programs are in use, in addition to public policies for the prevention and punishment of criminals. This paper proposes the use of pattern classifiers based on neuroevolution technique (NEAT) in tropical forest deforestation detection tasks.
  arXiv  Detail & Related papers  (2022-08-23T16:04:12Z)
• Explaining random forest prediction through diverse rulesets [0.0]
  Local Tree eXtractor (LTreeX) is able to explain the forest prediction for a given test instance with a few diverse rules. We show that our proposed approach substantially outperforms other explainable methods in terms of predictive performance.
  arXiv  Detail & Related papers  (2022-03-29T12:54:57Z)
• Anti-Concentrated Confidence Bonuses for Scalable Exploration [57.91943847134011]
  Intrinsic rewards play a central role in handling the exploration-exploitation trade-off. We introduce emphanti-concentrated confidence bounds for efficiently approximating the elliptical bonus. We develop a practical variant for deep reinforcement learning that is competitive with contemporary intrinsic rewards on Atari benchmarks.
  arXiv  Detail & Related papers  (2021-10-21T15:25:15Z)
• EE-Net: Exploitation-Exploration Neural Networks in Contextual Bandits [52.98326168071513]
  "EE-Net" is a neural-based bandit approach with a novel exploration strategy. We show that EE-Net achieves $mathcalO(sqrtTlog T)$ regret, which is tighter than existing state-of-the-art neural bandit algorithms.
  arXiv  Detail & Related papers  (2021-10-07T04:12:36Z)
• Minimax Rates for High-Dimensional Random Tessellation Forests [0.0]
  Mondrian forests is the first class of random forests for which minimax rates were obtained in arbitrary dimension. We show that a large class of random forests with general split directions also achieve minimax optimal convergence rates in arbitrary dimension.
  arXiv  Detail & Related papers  (2021-09-22T06:47:38Z)
• Growing Deep Forests Efficiently with Soft Routing and Learned Connectivity [79.83903179393164]
  This paper further extends the deep forest idea in several important aspects. We employ a probabilistic tree whose nodes make probabilistic routing decisions, a.k.a., soft routing, rather than hard binary decisions. Experiments on the MNIST dataset demonstrate that our empowered deep forests can achieve better or comparable performance than [1],[3].
  arXiv  Detail & Related papers  (2020-12-29T18:05:05Z)
• Learning Representations for Axis-Aligned Decision Forests through Input Perturbation [2.755007887718791]
  Axis-aligned decision forests have long been the leading class of machine learning algorithms. Despite their widespread use and rich history, decision forests to date fail to consume raw structured data. We present a novel but intuitive proposal to achieve representation learning for decision forests.
  arXiv  Detail & Related papers  (2020-07-29T11:56:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.