Gradient Optimization for Single-State RMDPs

• URL: http://arxiv.org/abs/2209.12295v1
• Date: Sun, 25 Sep 2022 18:50:02 GMT
• Title: Gradient Optimization for Single-State RMDPs
• Authors: Keith Badger
• Abstract summary: Modern problems such as autonomous driving, control of robotic components, and medical diagnostics have become increasingly difficult to solve analytically. Data-driven solutions are a strong option where there are problems with more dimensions of complexity than can be understood by people. Unfortunately, data-driven models often come with uncertainty in how they will perform in the worst of scenarios. In fields such as autonomous driving and medicine, the consequences of these failures could be catastrophic.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As modern problems such as autonomous driving, control of robotic components, and medical diagnostics have become increasingly difficult to solve analytically, data-driven decision-making has seen a large gain in interest. Where there are problems with more dimensions of complexity than can be understood by people, data-driven solutions are a strong option. Many of these methods belong to a subdivision of machine learning known as reinforcement learning. Unfortunately, data-driven models often come with uncertainty in how they will perform in the worst of scenarios. Since the solutions are not derived analytically many times, these models will fail unpredictably. In fields such as autonomous driving and medicine, the consequences of these failures could be catastrophic. Various methods are being explored to resolve this issue and one of them is known as adversarial learning. It pits two models against each other by having one model optimize its goals as the opposite of the other model's goals. This type of training has the potential to find models which perform reliably in complex and high stakes settings, although it is not certain when this type of training will work. The goal is to gain insight about when these types of models will reach stable solutions.

Related papers

• What matters when building vision-language models? [52.8539131958858]
  We develop Idefics2, an efficient foundational vision-language model with 8 billion parameters. Idefics2 achieves state-of-the-art performance within its size category across various multimodal benchmarks. We release the model (base, instructed, and chat) along with the datasets created for its training.
  arXiv  Detail & Related papers  (2024-05-03T17:00:00Z)
• Towards Learning Stochastic Population Models by Gradient Descent [0.0]
  We show that simultaneous estimation of parameters and structure poses major challenges for optimization procedures. We demonstrate accurate estimation of models but find that enforcing the inference of parsimonious, interpretable models drastically increases the difficulty.
  arXiv  Detail & Related papers  (2024-04-10T14:38:58Z)
• Multi-teacher knowledge distillation as an effective method for compressing ensembles of neural networks [0.0]
  Large-scale deep models have achieved great success, but the enormous computational complexity and gigantic storage requirements make them difficult to implement in real-time applications. We present a modified knowledge distillation framework which allows compressing the entire ensemble model into a weight space of a single model. We show that knowledge distillation can aggregate knowledge from multiple teachers in only one student model and, with the same computational complexity, obtain a better-performing model compared to a model trained in the standard manner.
  arXiv  Detail & Related papers  (2023-02-14T17:40:36Z)
• Dataless Knowledge Fusion by Merging Weights of Language Models [51.8162883997512]
  Fine-tuning pre-trained language models has become the prevalent paradigm for building downstream NLP models. This creates a barrier to fusing knowledge across individual models to yield a better single model. We propose a dataless knowledge fusion method that merges models in their parameter space.
  arXiv  Detail & Related papers  (2022-12-19T20:46:43Z)
• Winning solutions and post-challenge analyses of the ChaLearn AutoDL challenge 2019 [112.36155380260655]
  This paper reports the results and post-challenge analyses of ChaLearn's AutoDL challenge series. Results show that DL methods dominated, though popular Neural Architecture Search (NAS) was impractical. A high level modular organization emerged featuring a "meta-learner", "data ingestor", "model selector", "model/learner", and "evaluator"
  arXiv  Detail & Related papers  (2022-01-11T06:21:18Z)
• High-Accuracy Model-Based Reinforcement Learning, a Survey [2.0196229393131726]
  Deep reinforcement learning has shown remarkable success in game playing and robotics. To reduce the number of environment samples, model-based reinforcement learning creates an explicit model of the environment dynamics. Some of these methods succeed in achieving high accuracy at low sample complexity, most do so either in a robotics or in a games context.
  arXiv  Detail & Related papers  (2021-07-17T14:01:05Z)
• On the Efficacy of Adversarial Data Collection for Question Answering: Results from a Large-Scale Randomized Study [65.17429512679695]
  In adversarial data collection (ADC), a human workforce interacts with a model in real time, attempting to produce examples that elicit incorrect predictions. Despite ADC's intuitive appeal, it remains unclear when training on adversarial datasets produces more robust models.
  arXiv  Detail & Related papers  (2021-06-02T00:48:33Z)
• Sufficiently Accurate Model Learning for Planning [119.80502738709937]
  This paper introduces the constrained Sufficiently Accurate model learning approach. It provides examples of such problems, and presents a theorem on how close some approximate solutions can be. The approximate solution quality will depend on the function parameterization, loss and constraint function smoothness, and the number of samples in model learning.
  arXiv  Detail & Related papers  (2021-02-11T16:27:31Z)
• Diverse Complexity Measures for Dataset Curation in Self-driving [80.55417232642124]
  We propose a new data selection method that exploits a diverse set of criteria that quantize interestingness of traffic scenes. Our experiments show that the proposed curation pipeline is able to select datasets that lead to better generalization and higher performance.
  arXiv  Detail & Related papers  (2021-01-16T23:45:02Z)
• A Simple and Interpretable Predictive Model for Healthcare [0.0]
  Deep learning models are currently dominating most state-of-the-art solutions for disease prediction. These deep learning models, with trainable parameters running into millions, require huge amounts of compute and data to train and deploy. We develop a simpler yet interpretable non-deep learning based model for application to EHR data.
  arXiv  Detail & Related papers  (2020-07-27T08:13:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.