Combining Trained Models in Reinforcement Learning

• URL: http://arxiv.org/abs/2605.02159v1
• Date: Mon, 04 May 2026 02:37:55 GMT
• Title: Combining Trained Models in Reinforcement Learning
• Abstract summary: Deep reinforcement learning (DRL) has delivered strong results in domains such as Atari and Go, but it still suffers from high sample cost and weak transfer beyond the training setting.<n>A common response is to reuse information from previously trained models through transfer, distillation, ensemble methods, or federated training.<n>This paper presents a PRISMA-guided systematic review of empirical studies on pretrained knowledge reuse in DRL.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep reinforcement learning (DRL) has delivered strong results in domains such as Atari and Go, but it still suffers from high sample cost and weak transfer beyond the training setting. A common response is to reuse information from previously trained models through transfer, distillation, ensemble methods, or federated training instead of learning each target task from random initialization. The literature on these mechanisms is fragmented, and published comparisons are hard to interpret because tasks, baselines, and compute budgets differ. This paper presents a PRISMA-guided systematic review of empirical studies on pretrained knowledge reuse in DRL. Starting from 589 records retrieved from IEEE Xplore, the ACM Digital Library, and citation tracing, we screened 570 unique records and assessed 89 full texts. After applying the final eligibility criteria, 15 empirical studies remained in the main synthesis. We analyzed them qualitatively across three factors: source-target similarity, diversity among reused models, and the fairness of comparisons against from-scratch baselines. Three patterns recur across the surviving corpus. First, positive results are concentrated in settings where source and target tasks share substantial structure or where the method includes an explicit gating or alignment mechanism. Second, evidence for ensembles and federated aggregation is promising but sparse and mostly limited to narrow settings. Third, compute-matched comparisons are rare, which weakens claims about efficiency gains over stronger single-agent baselines. The paper contributes a narrower and internally consistent review scope, a study-level synthesis of empirical evidence, and a provisional independence spectrum that should be treated as a hypothesis for future benchmarking rather than a validated metric.

Related papers

• Dense Process Supervision for Search Agents via Fact Utility Estimation [39.89624659103789]
  We propose a dense process supervision method based on fact utility estimation.<n> Experiments on seven single-hop and multi-hop QA benchmarks show that our method consistently outperforms existing baselines.
  arXiv  Detail & Related papers  (2026-09-01T07:34:33Z)
• MultAttnAttrib: Training-Free Multimodal Attribution in Long Document Question Answering [73.47440561592556]
  We introduce MultAttnAttrib, a training-free attribution-generation method.<n>We show that MultAttnAttrib consistently outperforms a variety of attribution-generation methods.<n>Our method produces attributions at up to one-seventh of the direct inference latency compared to prompting.
  arXiv  Detail & Related papers  (2026-07-01T19:29:19Z)
• Retrieval, Reward, and Training Protocols: What Matters in Training Search Agents? [15.605151408619895]
  We present a controlled empirical study that isolates three under-explored dimensions of search agent training.<n>First, we identify a critical data-coverage issue in the widely used Wikipedia 2018 corpus and show that correcting it alone yields larger gains than the differences between training algorithms.<n>Second, we systematically compare outcome-based and process-based reward methods across three base models, finding that the simplest outcome-based approach achieves competitive or superior performance in most settings.<n>Third, we analyze training data diversity, off-policy data utilization, and search budget scaling, distilling practical guidelines for training effective search agents.
  arXiv  Detail & Related papers  (2026-05-27T03:04:36Z)
• Reducing Hallucinations in LLM-based Scientific Literature Analysis Using Peer Context Outlier Detection [15.38900394412587]
  This paper introduces Peer Context Outlier Detection (P-COD), a novel approach that uses the relationships between documents to improve extraction accuracy.<n>By comparing extracted data to validated peer information within the corpus, we adjust confidence scores and flag low-confidence results for expert review.<n>Our experiments demonstrate up to 98% precision in outlier detection across 6 domains of science, demonstrating that our design reduces hallucinations, enhances trust in automated systems, and allows researchers to focus on ambiguous cases.
  arXiv  Detail & Related papers  (2026-04-01T23:15:59Z)
• Trustworthiness Preservation by Copies of Machine Learning Systems [0.0]
  We introduce a calculus to model and verify complex queries over data.<n>We define four distinct notions: Justifiably, Equally, Weakly and Almost Trustworthy.<n>The aim is to offer a computational tool to check the trustworthiness of possibly complex systems copied from an original whose behavour is known.
  arXiv  Detail & Related papers  (2025-06-05T16:14:57Z)
• Enhancing Study-Level Inference from Clinical Trial Papers via Reinforcement Learning-Based Numeric Reasoning [12.050619995670038]
  We conceptualise the problem as one of quantitative reasoning.<n>We develop a numeric reasoning system composed of a numeric data extraction model and an effect estimate component.
  arXiv  Detail & Related papers  (2025-05-28T22:59:45Z)
• Evaluating Human Alignment and Model Faithfulness of LLM Rationale [66.75309523854476]
  We study how well large language models (LLMs) explain their generations through rationales. We show that prompting-based methods are less "faithful" than attribution-based explanations.
  arXiv  Detail & Related papers  (2024-06-28T20:06:30Z)
• Evaluating BERT-based Scientific Relation Classifiers for Scholarly Knowledge Graph Construction on Digital Library Collections [5.8962650619804755]
  Inferring semantic relations between related scientific concepts is a crucial step. BERT-based pre-trained models have been popularly explored for automatic relation classification. Existing methods are primarily evaluated on clean texts. To address these limitations, we started by creating OCR-noisy texts.
  arXiv  Detail & Related papers  (2023-05-03T17:32:16Z)
• Towards Realistic Low-resource Relation Extraction: A Benchmark with Empirical Baseline Study [51.33182775762785]
  This paper presents an empirical study to build relation extraction systems in low-resource settings. We investigate three schemes to evaluate the performance in low-resource settings: (i) different types of prompt-based methods with few-shot labeled data; (ii) diverse balancing methods to address the long-tailed distribution issue; and (iii) data augmentation technologies and self-training to generate more labeled in-domain data.
  arXiv  Detail & Related papers  (2022-10-19T15:46:37Z)
• COLO: A Contrastive Learning based Re-ranking Framework for One-Stage Summarization [84.70895015194188]
  We propose a Contrastive Learning based re-ranking framework for one-stage summarization called COLO. COLO boosts the extractive and abstractive results of one-stage systems on CNN/DailyMail benchmark to 44.58 and 46.33 ROUGE-1 score.
  arXiv  Detail & Related papers  (2022-09-29T06:11:21Z)
• Assaying Out-Of-Distribution Generalization in Transfer Learning [103.57862972967273]
  We take a unified view of previous work, highlighting message discrepancies that we address empirically. We fine-tune over 31k networks, from nine different architectures in the many- and few-shot setting.
  arXiv  Detail & Related papers  (2022-07-19T12:52:33Z)
• SAIS: Supervising and Augmenting Intermediate Steps for Document-Level Relation Extraction [51.27558374091491]
  We propose to explicitly teach the model to capture relevant contexts and entity types by supervising and augmenting intermediate steps (SAIS) for relation extraction. Based on a broad spectrum of carefully designed tasks, our proposed SAIS method not only extracts relations of better quality due to more effective supervision, but also retrieves the corresponding supporting evidence more accurately.
  arXiv  Detail & Related papers  (2021-09-24T17:37:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.