Related papers: Smooth Sequential Optimisation with Delayed Feedback

Related papers

Do Agent Optimizers Compound? A Continual-Learning Evaluation on Terminal-Bench 2.0 [53.71882250666667]
Most reported gains from agent-optimization methods are one-shot.<n>This does not test the setting that matters for deployed agents.<n>We compare three approaches to agent-harness optimization.
arXiv Detail & Related papers (2026-07-15T16:36:04Z)
Optimizing Visual Generative Models via Distribution-wise Rewards [57.46109819595665]
We present a novel framework that finetunes generative models using distribution-wise rewards, ensuring better alignment with real-world data distributions.<n>Our approach significantly improves FID-50K across various base models, from 5.77 for SiT and from 3.74 to 3.52 for EDM2.
arXiv Detail & Related papers (2026-07-02T15:08:56Z)
MARBLE: Multi-Aspect Reward Balance for Diffusion RL [71.6241143519038]
Reinforcement learning has become the dominant approach for aligning diffusion models with human preferences.<n>Existing practice deal with multiple rewards by training one specialist model per reward.<n>We propose MARBLE, a gradient-space optimization framework that maintains independent advantage estimators for each reward.
arXiv Detail & Related papers (2026-05-07T16:20:42Z)
Uncertainty-Aware Reward Discounting for Mitigating Reward Hacking [0.0]
Real-world objectives are often uncertain, context-dependent, and internally inconsistent.<n>This mismatch can lead to alignment failures such as reward hacking, over-optimization, and overconfident behavior.<n>We introduce a dual-source uncertainty-aware reward framework that explicitly models both uncertainty in value estimation and uncertainty in human preferences.
arXiv Detail & Related papers (2026-04-29T07:14:01Z)
Stepwise Penalization for Length-Efficient Chain-of-Thought Reasoning [66.22060690012512]
Large reasoning models improve with more test-time computation, but often overthink, producing unnecessarily long chains-of-thought that raise cost without improving accuracy.<n>We propose Step-wise Adaptive Penalization (SWAP), a fine-grained framework that allocates length reduction across steps based on intrinsic contribution.
arXiv Detail & Related papers (2026-02-27T20:23:59Z)
Distribution-Aware Reward Estimation for Test-Time Reinforcement Learning [10.571602226707473]
Test-time reinforcement learning (TTRL) enables large language models to self-improve on unlabeled inputs.<n>Most existing TTRL methods rely on majority voting (MV) over rollouts to produce deterministic rewards.<n>We propose Distribution-AwareReward Estimation (DARE) which shifts reward estimation from a single majority outcome to the full empirical rollout distribution.
arXiv Detail & Related papers (2026-01-29T14:48:02Z)
Forward versus Backward: Comparing Reasoning Objectives in Direct Preference Optimization [0.12891210250935145]
Large language models exhibit impressive reasoning capabilities yet frequently generate plausible but incorrect solutions.<n>This paper investigates the effect of training objective composition on reasoning reliability through Direct Preference Optimization.
arXiv Detail & Related papers (2026-01-12T04:46:27Z)
Optimizing LLM Code Suggestions: Feedback-Driven Timing with Lightweight State Bounds [0.24160362166617352]
Large Language Models (LLMs) have transformed code auto-completion by generating context-aware suggestions.<n>We propose an adaptive timing mechanism that dynamically adjusts the delay before offering a suggestion based on real-time developer feedback.
arXiv Detail & Related papers (2025-11-24T07:29:15Z)
Learning to Undo: Rollback-Augmented Reinforcement Learning with Reversibility Signals [1.9537983097153042]
This paper proposes a reversible learning framework to improve the robustness and efficiency of value based Reinforcement Learning agents.<n>The framework has two complementary core mechanisms: an empirically derived transition reversibility measure called Phi of s and a interrupts, and a selective state rollback operation.
arXiv Detail & Related papers (2025-10-16T09:48:54Z)
ReDit: Reward Dithering for Improved LLM Policy Optimization [6.841631032347429]
DeepSeek-R1 has successfully enhanced Large Language Model (LLM) reasoning capabilities through its rule-based reward system.<n>While it's a ''perfect'' reward system that effectively mitigates reward hacking, such reward functions are often discrete.<n>We propose ReDit (Reward Dithering), a method that dithers the discrete reward signal by adding simple random noise.
arXiv Detail & Related papers (2025-06-23T13:36:24Z)
Simulation-Based Inference for Adaptive Experiments [38.841210420855276]
Multi-arm bandit experimental designs are increasingly being adopted over standard randomized trials.<n>We propose a simulation-based approach for conducting hypothesis tests and constructing confidence intervals for arm specific means.<n>Our results show that our approach achieves the desired coverage while reducing confidence interval widths by up to 50%, with drastic improvements for arms not targeted by the design.
arXiv Detail & Related papers (2025-06-03T13:46:59Z)
Improving self-training under distribution shifts via anchored confidence with theoretical guarantees [13.796664304274643]
Self-training often falls short under distribution shifts due to an increased discrepancy between prediction confidence and actual accuracy. We develop a principled method to improve self-training under distribution shifts based on temporal consistency. Our approach consistently improves self-training performances by 8% to 16% across diverse distribution shift scenarios without a computational overhead.
arXiv Detail & Related papers (2024-11-01T13:48:11Z)
Overcoming Reward Overoptimization via Adversarial Policy Optimization with Lightweight Uncertainty Estimation [46.61909578101735]
Adversarial Policy Optimization (AdvPO) is a novel solution to the pervasive issue of reward over-optimization in Reinforcement Learning from Human Feedback. In this paper, we introduce a lightweight way to quantify uncertainties in rewards, relying solely on the last layer embeddings of the reward model.
arXiv Detail & Related papers (2024-03-08T09:20:12Z)
Augmenting Unsupervised Reinforcement Learning with Self-Reference [63.68018737038331]
Humans possess the ability to draw on past experiences explicitly when learning new tasks. We propose the Self-Reference (SR) approach, an add-on module explicitly designed to leverage historical information. Our approach achieves state-of-the-art results in terms of Interquartile Mean (IQM) performance and Optimality Gap reduction on the Unsupervised Reinforcement Learning Benchmark.
arXiv Detail & Related papers (2023-11-16T09:07:34Z)
Exploiting Correlated Auxiliary Feedback in Parameterized Bandits [56.84649080789685]
We study a novel variant of the parameterized bandits problem in which the learner can observe additional auxiliary feedback that is correlated with the observed reward. The auxiliary feedback is readily available in many real-life applications, e.g., an online platform that wants to recommend the best-rated services to its users can observe the user's rating of service (rewards) and collect additional information like service delivery time (auxiliary feedback)
arXiv Detail & Related papers (2023-11-05T17:27:06Z)
Did You Mean...? Confidence-based Trade-offs in Semantic Parsing [52.28988386710333]
We show how a calibrated model can help balance common trade-offs in task-oriented parsing. We then examine how confidence scores can help optimize the trade-off between usability and safety.
arXiv Detail & Related papers (2023-03-29T17:07:26Z)
Towards Reliable Uncertainty Quantification via Deep Ensembles in Multi-output Regression Task [0.0]
This study aims to investigate the deep ensemble approach, an approximate Bayesian inference, in the multi-output regression task. A trend towards underestimation of uncertainty as it increases is observed for the first time. We propose the deep ensemble framework that applies the post-hoc calibration method to improve its uncertainty quantification performance.
arXiv Detail & Related papers (2023-03-28T05:10:57Z)
Variational Refinement for Importance Sampling Using the Forward Kullback-Leibler Divergence [77.06203118175335]
Variational Inference (VI) is a popular alternative to exact sampling in Bayesian inference. Importance sampling (IS) is often used to fine-tune and de-bias the estimates of approximate Bayesian inference procedures. We propose a novel combination of optimization and sampling techniques for approximate Bayesian inference.
arXiv Detail & Related papers (2021-06-30T11:00:24Z)
Counterfactual Evaluation of Slate Recommendations with Sequential Reward Interactions [18.90946044396516]
Music streaming, video streaming, news recommendation, and e-commerce services often engage with content in a sequential manner. Providing and evaluating good sequences of recommendations is therefore a central problem for these services. We propose a new counterfactual estimator that allows for sequential interactions in the rewards with lower variance in anally unbiased manner.
arXiv Detail & Related papers (2020-07-25T17:58:01Z)
Extrapolation for Large-batch Training in Deep Learning [72.61259487233214]
We show that a host of variations can be covered in a unified framework that we propose. We prove the convergence of this novel scheme and rigorously evaluate its empirical performance on ResNet, LSTM, and Transformer.
arXiv Detail & Related papers (2020-06-10T08:22:41Z)
Adversarial Robustness on In- and Out-Distribution Improves Explainability [109.68938066821246]
RATIO is a training procedure for robustness via Adversarial Training on In- and Out-distribution. RATIO achieves state-of-the-art $l$-adrial on CIFAR10 and maintains better clean accuracy.
arXiv Detail & Related papers (2020-03-20T18:57:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.