Robust Vision-Based Runway Detection through Conformal Prediction and Conformal mAP

• URL: http://arxiv.org/abs/2505.16740v1
• Date: Thu, 22 May 2025 14:52:59 GMT
• Title: Robust Vision-Based Runway Detection through Conformal Prediction and Conformal mAP
• Abstract summary: We explore the use of conformal prediction to provide statistical uncertainty guarantees for runway detection in vision-based landing systems (VLS)<n>Using fine-tuned YOLOv5 and YOLOv6 models on aerial imagery, we apply conformal prediction to quantify localization reliability under user-defined risk levels.<n>Our results show that conformal prediction can improve the reliability of runway detection by quantifying uncertainty in a statistically sound way.
• Score: 1.3794573109655741
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We explore the use of conformal prediction to provide statistical uncertainty guarantees for runway detection in vision-based landing systems (VLS). Using fine-tuned YOLOv5 and YOLOv6 models on aerial imagery, we apply conformal prediction to quantify localization reliability under user-defined risk levels. We also introduce Conformal mean Average Precision (C-mAP), a novel metric aligning object detection performance with conformal guarantees. Our results show that conformal prediction can improve the reliability of runway detection by quantifying uncertainty in a statistically sound way, increasing safety on-board and paving the way for certification of ML system in the aerospace domain.

Related papers

• FLY-EVAL++: An Evidence-Driven Evaluation Protocol for Safety-Constrained Flight Prediction with Large Language Models [45.90905049052029]
  We propose FLY-EVAL++, an evidence-driven evaluation protocol for large language models in safety-critical environments.<n>We instantiate FLY-EVAL++ for Flight Trajectory and Attitude Prediction (FTAP) by extending the PilotBench setting with history-conditioned and multi-step prediction tasks.<n>Our results show that evaluation in safety-critical domains should measure constraint satisfaction and structured validity explicitly rather than rely on accuracy-centric reporting alone.
  arXiv  Detail & Related papers  (2026-09-03T16:00:02Z)
• Conformal Changepoint Localization and Root Cause Analysis with Corrupted Observations [55.76952683833966]
  This paper proposes weighted CONCH (W-CONCH) and weighted CROC (W-CROC) to reduce confidence set size when data may be corrupted.<n>Experiments on image-based and real-world changepoint and root-cause benchmarks show that uncertainty-based weighting substantially reduces confidence set size.
  arXiv  Detail & Related papers  (2026-07-29T05:16:59Z)
• Conformal Reliability: A New Evaluation Metric for Conditional Generation [62.761166941396844]
  Conditional generative models have recently remarkable success in various applications.<n>We propose a novel evaluation metric called reliability based on prediction set at a pre-specified confidence level.<n>We introduce Conformal ReLiability (CReL), a framework that can (i) construct a prediction set and (ii) accurately optimize the reliability score within the constructed prediction set.
  arXiv  Detail & Related papers  (2026-05-29T03:52:44Z)
• Safety Beyond the Training Data: Robust Out-of-Distribution MPC via Conformalized System Level Synthesis [3.5174884177930448]
  We present a novel framework for robust out-of-distribution planning and control using conformal prediction (CP) and system level synthesis (SLS)<n>We first derive high-confidence model error bounds using weighted CP with a learned, state-control-dependent covariance model.<n>These bounds are integrated into an SLS-based robust nonlinear model predictive control (MPC) formulation, which performs constraint tightening over the prediction horizon via volume-optimized forward reachable sets.
  arXiv  Detail & Related papers  (2026-02-12T15:11:44Z)
• Conformal Safety Monitoring for Flight Testing: A Case Study in Data-Driven Safety Learning [7.0366838632617705]
  We develop a data-driven approach for safety monitoring in flight testing, where pilots perform maneuvers on aircraft with uncertain parameters.<n>We use offline trajectory simulation to learn a calibrated statistical model of the short-term safety risk facing pilots.<n>We evaluate our method on a flight dynamics model with uncertain parameters, demonstrating its ability to reliably identify unsafe scenarios.
  arXiv  Detail & Related papers  (2025-11-25T19:57:07Z)
• Trust, or Don't Predict: Introducing the CWSA Family for Confidence-Aware Model Evaluation [0.0]
  We introduce two new metrics Confidence-Weighted Selective Accuracy (CWSA) and its normalized variant CWSA+.<n>CWSA offers principled and interpretable way to evaluate predictive models under confidence thresholds.<n>We show that CWSA and CWSA+ both effectively detect nuanced failure modes and outperform classical metrics in trust-sensitive tests.
  arXiv  Detail & Related papers  (2025-05-24T10:07:48Z)
• SConU: Selective Conformal Uncertainty in Large Language Models [59.25881667640868]
  We propose a novel approach termed Selective Conformal Uncertainty (SConU)<n>We develop two conformal p-values that are instrumental in determining whether a given sample deviates from the uncertainty distribution of the calibration set at a specific manageable risk level.<n>Our approach not only facilitates rigorous management of miscoverage rates across both single-domain and interdisciplinary contexts, but also enhances the efficiency of predictions.
  arXiv  Detail & Related papers  (2025-04-19T03:01:45Z)
• Uncertainty-Aware Online Extrinsic Calibration: A Conformal Prediction Approach [4.683612295430957]
  We present the first approach to integrate uncertainty awareness into online calibration, combining Monte Carlo Dropout with Conformal Prediction.<n>We demonstrate effectiveness across different visual sensor types, measuring performance with adapted metrics to evaluate the efficiency and reliability of the intervals.<n>We offer insights into the reliability of calibration estimates, which can greatly improve the robustness of sensor fusion in dynamic environments.
  arXiv  Detail & Related papers  (2025-01-12T17:24:51Z)
• Addressing Uncertainty in LLMs to Enhance Reliability in Generative AI [47.64301863399763]
  We present a dynamic semantic clustering approach inspired by the Chinese Restaurant Process. We quantify uncertainty of Large Language Models (LLMs) on a given query by calculating entropy of the generated semantic clusters. We propose leveraging the (negative) likelihood of these clusters as the (non)conformity score within Conformal Prediction framework.
  arXiv  Detail & Related papers  (2024-11-04T18:49:46Z)
• Confidence Aware Learning for Reliable Face Anti-spoofing [52.23271636362843]
  We propose a Confidence Aware Face Anti-spoofing model, which is aware of its capability boundary.<n>We estimate its confidence during the prediction of each sample.<n>Experiments show that the proposed CA-FAS can effectively recognize samples with low prediction confidence.
  arXiv  Detail & Related papers  (2024-11-02T14:29:02Z)
• Online scalable Gaussian processes with conformal prediction for guaranteed coverage [32.21093722162573]
  The consistency of the resulting uncertainty values hinges on the premise that the learning function conforms to the properties specified by the GP model. We propose to wed the GP with the prevailing conformal prediction (CP), a distribution-free post-processing framework that produces it prediction sets with a provably valid coverage.
  arXiv  Detail & Related papers  (2024-10-07T19:22:15Z)
• Spatial-Aware Conformal Prediction for Trustworthy Hyperspectral Image Classification [39.71307720326761]
  Hyperspectral image (HSI) classification involves assigning unique labels to each pixel to identify various land cover categories. Deep classifiers have achieved high predictive accuracy in this field, but they lack the ability to quantify confidence in their predictions. We introduce Spatial-Aware Conformal Prediction (textttSACP), a conformal prediction framework specifically designed for HSI data.
  arXiv  Detail & Related papers  (2024-09-02T13:11:38Z)
• ConU: Conformal Uncertainty in Large Language Models with Correctness Coverage Guarantees [68.33498595506941]
  We introduce a novel uncertainty measure based on self-consistency theory. We then develop a conformal uncertainty criterion by integrating the uncertainty condition aligned with correctness into the CP algorithm. Empirical evaluations indicate that our uncertainty measure outperforms prior state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-06-29T17:33:07Z)
• Certified Human Trajectory Prediction [66.1736456453465]
  We propose a certification approach tailored for trajectory prediction that provides guaranteed robustness.<n>To mitigate the inherent performance drop through certification, we propose a diffusion-based trajectory denoiser and integrate it into our method.<n>We demonstrate the accuracy and robustness of the certified predictors and highlight their advantages over the non-certified ones.
  arXiv  Detail & Related papers  (2024-03-20T17:41:35Z)
• Confident Object Detection via Conformal Prediction and Conformal Risk Control: an Application to Railway Signaling [0.0]
  We demonstrate the use of the conformal prediction framework to construct reliable predictors for detecting railway signals. Our approach is based on a novel dataset that includes images taken from the perspective of a train operator and state-of-the-art object detectors.
  arXiv  Detail & Related papers  (2023-04-12T08:10:13Z)
• Reliability-Aware Prediction via Uncertainty Learning for Person Image Retrieval [51.83967175585896]
  UAL aims at providing reliability-aware predictions by considering data uncertainty and model uncertainty simultaneously. Data uncertainty captures the noise" inherent in the sample, while model uncertainty depicts the model's confidence in the sample's prediction.
  arXiv  Detail & Related papers  (2022-10-24T17:53:20Z)
• Meta-Learning Priors for Safe Bayesian Optimization [72.8349503901712]
  We build on a meta-learning algorithm, F-PACOH, capable of providing reliable uncertainty quantification in settings of data scarcity. As core contribution, we develop a novel framework for choosing safety-compliant priors in a data-riven manner. On benchmark functions and a high-precision motion system, we demonstrate that our meta-learned priors accelerate the convergence of safe BO approaches.
  arXiv  Detail & Related papers  (2022-10-03T08:38:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.