Policy Optimality Measurement for Multi-Vehicle Decision-Making: From Extrinsic Indicators to Intrinsic Quality

• URL: http://arxiv.org/abs/2608.01133v1
• Date: Sun, 02 Aug 2026 10:21:58 GMT
• Title: Policy Optimality Measurement for Multi-Vehicle Decision-Making: From Extrinsic Indicators to Intrinsic Quality
• Abstract summary: This letter proposes a novel information-theoretic diagnostic framework to evaluate autonomous driving policies.<n>By leveraging a fully converged Monte Carlo Tree Search (MCTS) as an oracle, we establish a theoretical ground-truth baseline distribution.<n>We semantically decouple this metric into lateral and longitudinal dimensions, creating a granular "semantic microscope"
• Score: 13.870167545420935
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Evaluating Multi-Agent Reinforcement Learning (MARL) policies in autonomous driving fundamentally relies on extrinsic statistical indicators (e.g., reward curves and success rates), which often mask intrinsic policy degradation and algorithmic blind spots. To break this black-box evaluation, this letter proposes a novel information-theoretic diagnostic framework. By leveraging a fully converged Monte Carlo Tree Search (MCTS) as an asymptotic oracle, we establish a theoretical ground-truth baseline distribution. We formulate a bounded policy optimality score ($\mathcal{M}_{opt}$) using the forward KL divergence to rigorously penalize fatal collaborative omissions. Crucially, we semantically decouple this metric into lateral and longitudinal dimensions, creating a granular "semantic microscope". Extensive spatial and temporal diagnostics on state-of-the-art MARL architectures and exploration mechanisms demonstrate that our framework conclusively exposes hidden directional biases, identifies temporal average-policy traps, and transforms heuristic hyperparameter tuning into a visually trackable trajectory optimization. This framework establishes a rigorous, model-agnostic standard for benchmarking intrinsic multi-agent policy quality.

Related papers

• A Continuous-Time Reinforcement Learning Framework for Fine-Tuning Discrete Diffusion Models [9.133661763917374]
  We formulate reinforcement learning (RL) in continuous time with discrete state spaces and possibly arbitrary action spaces.<n>We develop a complete continuous-time RL framework for fine-tuning score-based discrete diffusion models.
  arXiv  Detail & Related papers  (2026-07-16T03:19:53Z)
• Expanding SPHERE-JEPA: A Family of Statistical Regularizers for the Hypersphere [11.087393998848254]
  In Self-Supervised Learning (SSL), preventing representation collapse by explicitly enforcing a uniform distribution on the unit hypersphere has proven to be effective.<n>Current frameworks typically rely on sliced statistical regularizers such as SIGReg (used in LeJEPA) and SUSReg (used in SPHEREJEPA)<n>This injects projection variance into the training gradients, destabilizing optimization, and hindering convergence.<n>In this work, we first show that analytically integrating out these random projections yields a deterministic Maximum Mean Discrepancy (MMD), bypassing the variance of sliced methods.
  arXiv  Detail & Related papers  (2026-06-16T07:10:05Z)
• Influence-Guided Symbolic Regression: Scientific Discovery via LLM-Driven Equation Search with Granular Feedback [56.69850045068714]
  We introduce textitInfluence-Guided Regression (IGSR), a method that frames equation discovery as an iterative two-step process.<n>We demonstrate IGSRs effectiveness on a diverse suite of benchmarks, including LLM-SRBench, pharmacological PKPD models, an epidemiological simulation, and real-world genomic data.
  arXiv  Detail & Related papers  (2026-05-27T23:48:01Z)
• Reinforcement Learning for Laser Additive Manufacturing Scan-Order Optimisation: A Bilevel Proxy--FEA Diagnostic Framework for Reward and World-Model Diagnosis [21.97902184672397]
  Reinforcement learning offers a promising approach for scan-order optimisation in laser additive manufacturing.<n>Full finite-element analysis is computationally prohibitive for dense in-the-loop evaluation.<n>This paper investigates a bilevel Proxy--FEA diagnostic framework for reward and world-model diagnosis.
  arXiv  Detail & Related papers  (2026-05-24T13:17:05Z)
• Regulating Anatomy-Aware Rewards via Trajectory-Integral Feedback for Volumetric Computed Tomography Analysis [30.439440028190944]
  We introduce textbfClinical Abnormality Benchmarking Substrate (CABS), a structured system that decomposes radiology reports into verifiable clinical semantic units.<n>We propose textbfTrajectory-Integral FeedbackO (TIF-GRPO), a novel framework integrating control-theoretic principles into policy optimization.
  arXiv  Detail & Related papers  (2026-05-19T04:33:27Z)
• Split the Differences, Pool the Rest: Provably Efficient Multi-Objective Imitation [49.86232017439639]
  We introduce Multi-Output Augmented Behavioral Cloning (MA-BC)<n>MA-BC partitions divergent expert data while pooling state-action pairs where no behavior conflict is observed.<n>We establish a novel lower bound for multi-objective imitation learning, demonstrating that MA-BC is minimax optimal.
  arXiv  Detail & Related papers  (2026-05-12T11:49:08Z)
• Too Correct to Learn: Reinforcement Learning on Saturated Reasoning Data [55.84428098924793]
  We propose Constrained Uniform Top-K Sampling (CUTS), a parameter-free decoding strategy enforcing structure-preserving exploration.<n>We integrate this into Mixed-CUTS, a training framework synergizing exploitative and exploratory rollouts to amplify intra-group advantage variance.<n> Notably, Mixed-CUTS improves Pass@1 accuracy on the challenging AIME25 benchmark by up to 15.1% over standard GRPO.
  arXiv  Detail & Related papers  (2026-04-20T16:43:28Z)
• Blind-Spot Mass: A Good-Turing Framework for Quantifying Deployment Coverage Risk in Machine Learning Systems [0.0]
  Blindspot mass is a Good-Turing framework for quantifying deployment coverage risk in machine learning.<n>We propose blind-spot mass B_n(tau), a deployment metric estimating the total probability mass assigned to states whose empirical support falls below a threshold tau.
  arXiv  Detail & Related papers  (2026-04-06T18:06:38Z)
• Gaussian Joint Embeddings For Self-Supervised Representation Learning [0.0]
  Self-parametric representation learning often relies on deterministic predictive architectures to align context and target views in latent space.<n>We propose a probabilistic alternative based on generative joint modeling.
  arXiv  Detail & Related papers  (2026-03-26T00:54:54Z)
• SIGMA: Scalable Spectral Insights for LLM Collapse [51.863164847253366]
  We introduce SIGMA (Spectral Inequalities for Gram Matrix Analysis), a unified framework for model collapse.<n>By utilizing benchmarks that deriving and deterministic bounds on the matrix's spectrum, SIGMA provides a mathematically grounded metric to track the contraction of the representation space.<n>We demonstrate that SIGMA effectively captures the transition towards states, offering both theoretical insights into the mechanics of collapse.
  arXiv  Detail & Related papers  (2026-01-06T19:47:11Z)
• Statistical control for spatio-temporal MEG/EEG source imaging with desparsified multi-task Lasso [102.84915019938413]
  Non-invasive techniques like magnetoencephalography (MEG) or electroencephalography (EEG) offer promise of non-invasive techniques. The problem of source localization, or source imaging, poses however a high-dimensional statistical inference challenge. We propose an ensemble of desparsified multi-task Lasso (ecd-MTLasso) to deal with this problem.
  arXiv  Detail & Related papers  (2020-09-29T21:17:16Z)
• Global Convergence of Policy Gradient for Linear-Quadratic Mean-Field Control/Game in Continuous Time [109.06623773924737]
  We study the policy gradient method for the linear-quadratic mean-field control and game. We show that it converges to the optimal solution at a linear rate, which is verified by a synthetic simulation.
  arXiv  Detail & Related papers  (2020-08-16T06:34:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.