Risk-Aware Online Conformal State Probing
Abstract Overview
The paper studies a sequential decision-making setting in which a centralized agent must choose both control actions and whether to probe the true state from a robot or edge device. It introduces Online Conformal State Probing (OCSP), a wrapper around an arbitrary pre-trained state predictor or value-based controller that uses belief-dependent outage estimates to decide when to query the state. The method is designed to control missed query error (MQE), defined as the fraction of outage events for which probing would have been beneficial but was not performed, while also trying to reduce the probing rate. The paper provides a high-probability worst-case MQE guarantee without distributional assumptions and validates the method in simulated Markov decision processes under exact, overconfident, and underconfident belief models.
Novelty
The main novelty is the use of online conformal risk-control ideas to govern active state probing in a closed-loop control problem, rather than calibrating a predictor or deferring decisions. OCSP is distinctive in that it can wrap existing pre-trained value-based policies and certify worst-case reliability of probing decisions without retraining or distributional assumptions.
Results
The experiments show that OCSP controls MQE to the target level (illustrated for α = 0.3) across belief models with different calibration quality, while the theoretical upper bound also bounds the observed behavior. In the reported comparison, an augment-and-train baseline matched to a similar probing rate attains MQE as high as 0.4, whereas OCSP maintains the target MQE. The study also shows probing-rate and return trade-offs: OCSP tends to probe more early on and then stabilizes, and belief calibration affects the balance between query rate and cumulative return.
Key Points
- OCSP estimates outage risk from the current belief and probes with a thresholded stochastic rule that includes forced exploration, then updates the threshold online using observed outages.
- The method comes with a high-probability upper bound on missed query error for any state sequence and fixed belief functions, giving worst-case reliability guarantees without distributional assumptions.
- In simulated MDP experiments, OCSP meets the target MQE across exact, overconfident, and underconfident belief models, while showing different probing-rate versus return trade-offs and competitive behavior against an augment-and-train baseline.