GRPO-QM: Target Preserving Exploration for Quantum Tomography

• URL: http://arxiv.org/abs/2609.14711v1
• Date: Sun, 13 Sep 2026 18:09:43 GMT
• Title: GRPO-QM: Target Preserving Exploration for Quantum Tomography
• Abstract summary: GRPO-QM sidesteps this by learning only an exploration strategy for a stated quantum-tomography posterior.<n>We then examine what learning contributes beyond physical proposal mechanisms and prior knowledge.
• Score: 42.21989596721942
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reward-based learning can alter the very posterior distribution that scientific inference aims to estimate. GRPO-QM sidesteps this by learning only an exploration strategy for a stated quantum-tomography posterior: a group-relative policy chooses among reversible physical moves, and an exact Metropolis correction ensures the posterior remains stationary once the policy is fixed. We then examine what learning contributes beyond physical proposal mechanisms and prior knowledge. Reconstruction comparisons suggest that most of the gains over the tested flows come from those two components rather than from learning itself, and a closed-form counterexample explains why: a reward tied to accepted motion can increase even when a physical observable remains highly correlated. Reward comparisons that control for initialization also show that aggregating across posteriors can invert the ranking that a reward implies within any single posterior. Lastly, on 45 enumerated posteriors with three training seeds each, exact and sampled gradients of the same trajectory objective reduce mean physical estimation variance by $12.63\%$ and $6.32\%$ relative to a tuned mixture, while averaging the trajectory score without rescaling its penalty yields only $0.43\%$. This indicates that objective scaling accounts for part of the disparity between sampled and exact training. The regained benefit is concentrated at four shots and flips sign at sixteen, so these exact, visible-bank diagnostics identify a concrete, reproducible failure mode in sampled training, the objective-scaling fix that addresses part of it, and the remaining gap.

Related papers

• Query-Conditioned Spherical Centroid Aggregation for Multimodal Retrieval [5.4018351446007875]
  Multimodal retrieval integrates video, audio, subtitles, and text.<n>Recent geometric aggregators treat all modalities symmetrically.<n>SCALAR is a query-conditioned aggregator that assigns relevance-based weights to each available modality.
  arXiv  Detail & Related papers  (2026-09-14T10:21:23Z)
• On-Policy Self-Distillation in Diffusion Models [89.32656931795293]
  Reinforcement learning can align diffusion models with human preferences and task-specific objectives, but endpoint rewards do not specify how an intermediate denoising prediction should change.<n>We introduce DiffusionOPSD as an on-policy self-distillation framework that converts image-level reward guidance into explicit targets for clean-output predictions at sampled queries.<n>Across SD 3.5-M and the step-distilled Z-Image-Turbo, our approach achieves the best final held-out scores in 19 of 20 reward-matched settings across two backbones and ten evaluators.
  arXiv  Detail & Related papers  (2026-08-25T14:55:24Z)
• Learn What's Left, Not What's Mastered: Saturation Aware Advantage Reweighting for Multi-Reward Policy Optimization [60.89431018071735]
  Reinforcement learning with group-relative advantages has become the de facto standard for post-training language model reasoners.<n>We show that rollouts with distinct reward profiles can receive identical advantages, and all objectives are optimized with fixed relative weights regardless of their current level of saturation.<n>We introduce saturation-aware reweighting, which standardizes each reward objective independently and adaptively discounts its contribution according to a batch-level estimate of objective saturation.
  arXiv  Detail & Related papers  (2026-08-17T04:07:50Z)
• Information-Calibrated Quantum Diffusion: Aligning Forward Noise with Reverse Recoverability [10.481986290704821]
  We introduce the classical--quantum information decrement $_t=I(X:Q_t-1)-I(X:Q_t) as an intrinsic diffusion coordinate.<n>Along depolarization, equalizing $_t$ yields the unique minimax discretization of the forward information loss.
  arXiv  Detail & Related papers  (2026-08-14T08:44:51Z)
• SafeExplorer: An Unbiased Policy Gradient for Reinforcement Learning with Recovery Interventions [17.763106379903228]
  Training reinforcement-learning agents directly on physical robots makes every fall costly, since a fall can damage the platform and cannot be undone like a simulator reset.<n>A standard mitigation hands control to a separate recovery policy whenever the agent leaves a designer-specified safe region.<n>We address this bias with a drop-in modification of proximal policy optimization (PPO)<n>Our algorithm reduces training-time falls by factors of 233x, 48x, and 26x on HalfCheetah, Ant, and Unitree Go1 over standard PPO.
  arXiv  Detail & Related papers  (2026-07-09T20:41:45Z)
• Explaining RhythmFormer: A Systematic XAI Analysis of Periodic Sparse Attention for Remote Photoplethysmography [1.647210198730602]
  Photoplethys transformers achieve low heart-rate error on benchmarks, yet their decisions remain opaque.<n>Existing XAI is dominated by qualitative heatmap inspection without quantitative faithfulness metrics or physiology-grounded validation.<n>We adapt four methods (raw attention, rollout, flow, Beyond Intuition) to quantifying RhythmFormer's bi-level attention with top-$k$ selection.
  arXiv  Detail & Related papers  (2026-06-11T19:10:06Z)
• ARBOR: Online Process Rewards via a Reusable Rubric Buffer for Search Agents [48.80766702702854]
  LLM-based search agents are trained predominantly with outcome-only reward, leaving the search process itself unsupervised.<n>This signal degenerates on outcome-homogeneous groups where all sampled trajectories share the same correctness, yielding zero within-group advantage and no gradient.<n>We propose ARBOR (Adaptive Buffer for Online Reward), a reusable process-reward framework that maintains a rubric memory shared across queries.
  arXiv  Detail & Related papers  (2026-06-02T06:58:54Z)
• Focal Reward: Balanced Reinforcement Learning under Rubric-Based Rewards [41.19082300691048]
  We propose Focal Reward, a novel objective to balance the training of reinforcement learning under rubric-based rewards.<n>Experiments across three model scales and six benchmarks demonstrate that our Focal Reward method outperforms the strongest static aggregation baseline.
  arXiv  Detail & Related papers  (2026-05-26T05:50:46Z)
• Posterior Coreset Construction with Kernelized Stein Discrepancy for Model-Based Reinforcement Learning [78.30395044401321]
  We develop a novel model-based approach to reinforcement learning (MBRL) It relaxes the assumptions on the target transition model to belong to a generic family of mixture models. It can achieve up-to 50 percent reduction in wall clock time in some continuous control environments.
  arXiv  Detail & Related papers  (2022-06-02T17:27:49Z)
• Semi-supervised Contrastive Learning with Similarity Co-calibration [72.38187308270135]
  We propose a novel training strategy, termed as Semi-supervised Contrastive Learning (SsCL) SsCL combines the well-known contrastive loss in self-supervised learning with the cross entropy loss in semi-supervised learning. We show that SsCL produces more discriminative representation and is beneficial to few shot learning.
  arXiv  Detail & Related papers  (2021-05-16T09:13:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.