Related papers: Incentivizing Permissionless Distributed Learning of LLMs

Related papers

Beyond Binary Rewards: A Comparative Study of Reward Design for Reinforcement Unlearning [18.91502176461499]
Machine unlearning seeks to remove specific knowledge from trained language models without full retraining.<n>Recent work has reformulated unlearning as a Reinforcement Learning with Verifiable Rewards (RLVR) problem.<n>We study how reward design affects unlearning efficiency within the Reinforcement Unlearning framework.
arXiv Detail & Related papers (2026-07-30T10:15:59Z)
TRACE: A Unified Rollout Budget Allocation Framework for Efficient Agentic Reinforcement Learning [52.67756371231985]
Reinforcement learning with verifiable rewards (RLVR) is a promising approach for enhancing reasoning and agentic behavior in large language models.<n>We introduce Tree Rollout Allocation for Contrastive Exploration (TRACE), a unified rollout allocation framework.<n>Technically, TRACE allocates rollout budget to both prompt roots and intermediate prefixes that are most likely to yield mixed terminal rewards.
arXiv Detail & Related papers (2026-06-09T17:16:03Z)
MARBLE: Multi-Aspect Reward Balance for Diffusion RL [71.6241143519038]
Reinforcement learning has become the dominant approach for aligning diffusion models with human preferences.<n>Existing practice deal with multiple rewards by training one specialist model per reward.<n>We propose MARBLE, a gradient-space optimization framework that maintains independent advantage estimators for each reward.
arXiv Detail & Related papers (2026-05-07T16:20:42Z)
Self-Distillation Zero: Self-Revision Turns Binary Rewards into Dense Supervision [50.61441331643804]
Reinforcement learning (RLVR) relies on binary rewards, which are broadly applicable and powerful, but provide only sparse supervision during training.<n>Distillation provides dense token-level supervision, typically obtained from an external teacher or using high-quality demonstrations.<n>We propose Self-Distillation Zero (SD-Zero), a method that is substantially more training sample-efficient than RL and does not require an external teacher or high-quality demonstrations.
arXiv Detail & Related papers (2026-04-13T19:46:55Z)
Mitigating Reward Hacking in RLHF via Bayesian Non-negative Reward Modeling [49.41422138354821]
We propose a principled reward modeling framework that integrates non-negative factor analysis into the Bradley-Terry preference model.<n>BNRM represents rewards through a sparse, non-negative latent factor generative process.<n>We show that BNRM substantially mitigates reward over-optimization, improves robustness under distribution shifts, and yields more interpretable reward decompositions than strong baselines.
arXiv Detail & Related papers (2026-02-11T08:14:11Z)
Trust-Based Incentive Mechanisms in Semi-Decentralized Federated Learning Systems [0.0]
In federated learning (FL), decentralized model training allows multi-ple participants to collaboratively improve a shared machine learning model without exchanging raw data.<n>This paper proposes a novel trust-based incentive mechanism designed to evaluate and reward the quality of contributions in FL systems.
arXiv Detail & Related papers (2026-02-09T05:47:51Z)
LaSeR: Reinforcement Learning with Last-Token Self-Rewarding [54.72617309922891]
Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a core paradigm for enhancing the reasoning capabilities of Large Language Models (LLMs)<n>Previous practice requires the LLM to sequentially generate solutions and self-verifications using two separate prompt templates, which significantly reduces efficiency.<n>We propose LaSeR (Reinforcement Learning with Last-Token Self-Rewarding), an algorithm that simply augments the original RLVR loss with a MSE loss.
arXiv Detail & Related papers (2025-10-16T17:55:11Z)
Reinforcement Learning with Verifiable yet Noisy Rewards under Imperfect Verifiers [90.50039419576807]
Reinforcement Learning with Verifiable Rewards (RLVR) trains policies against automated verifiers to avoid costly human labeling.<n>To reduce vulnerability to verifier hacking, many RLVR systems collapse rewards to binary $0,1$ during training.<n>This choice carries a cost: it introduces textitfalse negatives (rejecting correct answers, FNs) and textitfalse positives (accepting incorrect ones, FPs)
arXiv Detail & Related papers (2025-10-01T13:56:44Z)
Blockchain-based Framework for Scalable and Incentivized Federated Learning [0.820828081284034]
Federated Learning (FL) enables collaborative model training without sharing raw data, preserving privacy while harnessing distributed datasets.<n>Traditional FL systems often rely on centralized aggregating mechanisms, introducing trust issues, single points of failure, and limited mechanisms for incentivizing meaningful client contributions.<n>This paper presents a blockchain-based FL framework that addresses these limitations by integrating smart contracts and a novel hybrid incentive mechanism.
arXiv Detail & Related papers (2025-02-20T00:38:35Z)
Proof-of-Collaborative-Learning: A Multi-winner Federated Learning Consensus Algorithm [2.5203968759841158]
We propose Proof-of-Collaborative-Learning (PoCL), a multi-winner federated learning validated consensus mechanism. PoCL redirects the power of blockchains to train federated learning models. We present a novel evaluation mechanism to ensure the efficiency of the locally trained models of miners.
arXiv Detail & Related papers (2024-07-17T21:14:05Z)
Dense Reward for Free in Reinforcement Learning from Human Feedback [64.92448888346125]
We leverage the fact that the reward model contains more information than just its scalar output. We use these attention weights to redistribute the reward along the whole completion. Empirically, we show that it stabilises training, accelerates the rate of learning, and, in practical cases, may lead to better local optima.
arXiv Detail & Related papers (2024-02-01T17:10:35Z)
Noisy Correspondence Learning with Self-Reinforcing Errors Mitigation [63.180725016463974]
Cross-modal retrieval relies on well-matched large-scale datasets that are laborious in practice. We introduce a novel noisy correspondence learning framework, namely textbfSelf-textbfReinforcing textbfErrors textbfMitigation (SREM)
arXiv Detail & Related papers (2023-12-27T09:03:43Z)
Unified Classification and Rejection: A One-versus-All Framework [47.58109235690227]
We build a unified framework for building open set classifiers for both classification and OOD rejection. By decomposing the $ K $-class problem into $ K $ one-versus-all (OVA) binary classification tasks, we show that combining the scores of OVA classifiers can give $ (K+1) $-class posterior probabilities. Experiments on popular OSR and OOD detection datasets demonstrate that the proposed framework, using a single multi-class classifier, yields competitive performance.
arXiv Detail & Related papers (2023-11-22T12:47:12Z)
Fair yet Asymptotically Equal Collaborative Learning [32.588043205577435]
In collaborative learning with streaming data, nodes jointly and continuously learn a machine learning (ML) model by sharing the latest model updates computed from their latest streaming data. This paper explores an incentive design that guarantees fairness so that nodes receive rewards commensurate to their contributions. We empirically demonstrate in two settings with real-world streaming data, that our proposed approach outperforms existing baselines in fairness and learning performance while remaining competitive in preserving equality.
arXiv Detail & Related papers (2023-06-09T08:57:14Z)
Distributional Reinforcement Learning with Dual Expectile-Quantile Regression [51.87411935256015]
quantile regression approach to distributional RL provides flexible and effective way of learning arbitrary return distributions.<n>We show that distributional estimation guarantees vanish, and we empirically observe that the estimated distribution rapidly collapses to its mean.<n>Motivated by the efficiency of $L$-based learning, we propose to jointly learn expectiles and quantiles of the return distribution in a way that allows efficient learning.
arXiv Detail & Related papers (2023-05-26T12:30:05Z)
Distributional Reward Estimation for Effective Multi-Agent Deep Reinforcement Learning [19.788336796981685]
We propose a novel Distributional Reward Estimation framework for effective Multi-Agent Reinforcement Learning (DRE-MARL) Our main idea is to design the multi-action-branch reward estimation and policy-weighted reward aggregation for stabilized training. The superiority of the DRE-MARL is demonstrated using benchmark multi-agent scenarios, compared with the SOTA baselines in terms of both effectiveness and robustness.
arXiv Detail & Related papers (2022-10-14T08:31:45Z)
Simultaneous Double Q-learning with Conservative Advantage Learning for Actor-Critic Methods [133.85604983925282]
We propose Simultaneous Double Q-learning with Conservative Advantage Learning (SDQ-CAL) Our algorithm realizes less biased value estimation and achieves state-of-the-art performance in a range of continuous control benchmark tasks.
arXiv Detail & Related papers (2022-05-08T09:17:16Z)
FreeTickets: Accurate, Robust and Efficient Deep Ensemble by Training with Dynamic Sparsity [74.58777701536668]
We introduce the FreeTickets concept, which can boost the performance of sparse convolutional neural networks over their dense network equivalents by a large margin. We propose two novel efficient ensemble methods with dynamic sparsity, which yield in one shot many diverse and accurate tickets "for free" during the sparse training process.
arXiv Detail & Related papers (2021-06-28T10:48:20Z)
Reward-Based 1-bit Compressed Federated Distillation on Blockchain [14.365210947456209]
Recent advent of various forms of Federated Knowledge Distillation (FD) paves the way for a new generation of robust and communication-efficient Federated Learning (FL) We introduce a novel decentralized federated learning framework where heavily compressed 1-bit soft-labels are aggregated on a smart contract. In a context where workers' contributions are now easily comparable, we modify the Peer Truth Serum for Crowdsourcing mechanism (PTSC) for FD to reward honest participation.
arXiv Detail & Related papers (2021-06-27T15:51:04Z)
Training Generative Adversarial Networks in One Stage [58.983325666852856]
We introduce a general training scheme that enables training GANs efficiently in only one stage. We show that the proposed method is readily applicable to other adversarial-training scenarios, such as data-free knowledge distillation.
arXiv Detail & Related papers (2021-02-28T09:03:39Z)
2CP: Decentralized Protocols to Transparently Evaluate Contributivity in Blockchain Federated Learning Environments [9.885896204530878]
We introduce 2CP, a framework comprising two novel protocols for Federated Learning. Crowdsource Protocol allows an actor to bring a model forward for training, and use their own data to evaluate the contributions made to it. The Consortium Protocol gives trainers the same guarantee even when no party owns the initial model and no dataset is available.
arXiv Detail & Related papers (2020-11-15T12:59:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.