Risk Controlled Image Retrieval

• URL: http://arxiv.org/abs/2307.07336v3
• Date: Mon, 06 Jan 2025 10:00:15 GMT
• Title: Risk Controlled Image Retrieval
• Abstract summary: We present Risk Controlled Image Retrieval (RCIR), which generates retrieval sets with coverage guarantee.<n>RCIR can be easily integrated with existing uncertainty-aware image retrieval systems.<n>The validity and efficiency of RCIR are demonstrated on four real-world datasets.
• Score: 15.60341568367361
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Most image retrieval research prioritizes improving predictive performance, often overlooking situations where the reliability of predictions is equally important. The gap between model performance and reliability requirements highlights the need for a systematic approach to analyze and address the risks associated with image retrieval. Uncertainty quantification technique can be applied to mitigate this issue by assessing uncertainty for retrieval sets, but it provides only a heuristic estimate of uncertainty rather than a guarantee. To address these limitations, we present Risk Controlled Image Retrieval (RCIR), which generates retrieval sets with coverage guarantee, i.e., retrieval sets that are guaranteed to contain the true nearest neighbors with a predefined probability. RCIR can be easily integrated with existing uncertainty-aware image retrieval systems, agnostic to data distribution and model selection. To the best of our knowledge, this is the first work that provides coverage guarantees to image retrieval. The validity and efficiency of RCIR are demonstrated on four real-world datasets: CAR-196, CUB-200, Pittsburgh, and ChestX-Det.

Related papers

• A Step Forward Towards Trustworthy Risk-Aware Facial Retrieval (RA-FR) [0.6445605125467574]
  Risk-Aware Facial Retrieval (RA-FR) is a framework that moves beyond fixed Top-$k$ retrieval to adaptive set generation.<n> RA-FR consistently satisfies a 5% risk target with an average retrieval set size of approximately 10 images.
  arXiv  Detail & Related papers  (2026-07-09T14:55:45Z)
• Conformal Reliability: A New Evaluation Metric for Conditional Generation [62.761166941396844]
  Conditional generative models have recently remarkable success in various applications.<n>We propose a novel evaluation metric called reliability based on prediction set at a pre-specified confidence level.<n>We introduce Conformal ReLiability (CReL), a framework that can (i) construct a prediction set and (ii) accurately optimize the reliability score within the constructed prediction set.
  arXiv  Detail & Related papers  (2026-05-29T03:52:44Z)
• Cycle-Consistent Search: Question Reconstructability as a Proxy Reward for Search Agent Training [80.20022221643414]
  Cycle-Consistent Search is a gold-supervision-free framework for training search agents.<n>We show that CCS achieves performance comparable to supervised baselines.<n>These results suggest that CCS provides a scalable training paradigm for training search agents in settings where gold supervision is unavailable.
  arXiv  Detail & Related papers  (2026-04-14T17:00:18Z)
• Optimized Certainty Equivalent Risk-Controlling Prediction Sets [29.939616090267975]
  This paper introduces optimized certainty equivalent RCPS (OCE-RCPS), a novel framework that provides high-probability guarantees on general optimized certainty equivalent (OCE) risk measures.<n> Experiments on image segmentation demonstrate that OCE-RCPS consistently meets target satisfaction rates across various risk measures and reliability configurations.
  arXiv  Detail & Related papers  (2026-02-14T08:03:27Z)
• Heterogeneous Uncertainty-Guided Composed Image Retrieval with Fine-Grained Probabilistic Learning [49.28548464288051]
  Composed Image Retrieval (CIR) enables image search by combining a reference image with modification text.<n>In intrinsic noise in CIR triplets incurs intrinsic uncertainty and threatens the model's robustness.<n>This paper introduces a Heterogeneous Uncertainty-Guided (HUG) paradigm to overcome these limitations.
  arXiv  Detail & Related papers  (2026-01-16T16:05:49Z)
• BAPO: Boundary-Aware Policy Optimization for Reliable Agentic Search [72.87861928940929]
  Boundary-Aware Policy Optimization (BAPO) is a novel RL framework designed to cultivate reliable boundary awareness without compromising accuracy.<n>BAPO introduces two key components: (i) a group-based boundary-aware reward that encourages an IDK response only when the reasoning reaches its limit, and (ii) an adaptive reward modulator that strategically suspends this reward during early exploration, preventing the model from exploiting IDK as a shortcut.
  arXiv  Detail & Related papers  (2026-01-16T07:06:58Z)
• ReliabilityRAG: Effective and Provably Robust Defense for RAG-based Web-Search [69.60882125603133]
  We present ReliabilityRAG, a framework for adversarial robustness that explicitly leverages reliability information of retrieved documents.<n>Our work is a significant step towards more effective, provably robust defenses against retrieved corpus corruption in RAG.
  arXiv  Detail & Related papers  (2025-09-27T22:36:42Z)
• COIN: Uncertainty-Guarding Selective Question Answering for Foundation Models with Provable Risk Guarantees [51.5976496056012]
  COIN is an uncertainty-guarding selection framework that calibrates statistically valid thresholds to filter a single generated answer per question.<n>COIN estimates the empirical error rate on a calibration set and applies confidence interval methods to establish a high-probability upper bound on the true error rate.<n>We demonstrate COIN's robustness in risk control, strong test-time power in retaining admissible answers, and predictive efficiency under limited calibration data.
  arXiv  Detail & Related papers  (2025-06-25T07:04:49Z)
• SUMI-IFL: An Information-Theoretic Framework for Image Forgery Localization with Sufficiency and Minimality Constraints [66.85363924364628]
  Image forgery localization (IFL) is a crucial technique for preventing tampered image misuse and protecting social safety. We introduce a novel information-theoretic IFL framework named SUMI-IFL that imposes sufficiency-view and minimality-view constraints on forgery feature representation.
  arXiv  Detail & Related papers  (2024-12-13T09:08:02Z)
• Uncertainty-Aware Regularization for Image-to-Image Translation [2.5274064055508174]
  We propose a method to improve uncertainty estimation in medical Image-to-Image (I2I) translation. Our model integrates aleatoric uncertainty and employs Uncertainty-Aware Regularization (UAR) inspired by simple priors to refine uncertainty estimates.
  arXiv  Detail & Related papers  (2024-11-24T14:05:27Z)
• Controlling Risk of Retrieval-augmented Generation: A Counterfactual Prompting Framework [77.45983464131977]
  We focus on how likely it is that a RAG model's prediction is incorrect, resulting in uncontrollable risks in real-world applications. Our research identifies two critical latent factors affecting RAG's confidence in its predictions. We develop a counterfactual prompting framework that induces the models to alter these factors and analyzes the effect on their answers.
  arXiv  Detail & Related papers  (2024-09-24T14:52:14Z)
• Gallery-Aware Uncertainty Estimation For Open-Set Face Recognition [2.6862667248315386]
  In open-set face recognition, one seeks to classify an image, which could also be unknown. Here, the low variance of probabilistic embedding does not imply a low error probability. We propose a method aware of two sources of ambiguity in the open-set recognition system.
  arXiv  Detail & Related papers  (2024-08-26T12:44:17Z)
• X-Fake: Juggling Utility Evaluation and Explanation of Simulated SAR Images [49.546627070454456]
  The distribution inconsistency between real and simulated data is the main obstacle that influences the utility of simulated SAR images. We propose a novel trustworthy utility evaluation framework with a counterfactual explanation for simulated SAR images for the first time, denoted as X-Fake. The proposed framework is validated on four simulated SAR image datasets obtained from electromagnetic models and generative artificial intelligence approaches.
  arXiv  Detail & Related papers  (2024-07-28T09:27:53Z)
• On the Estimation of Image-matching Uncertainty in Visual Place Recognition [7.769607568805291]
  In Visual Place Recognition (VPR) the pose of a query image is estimated by comparing the image to a map of reference images with known reference poses. This work compares for the first time the main approaches for estimating the image-matching uncertainty. We formulate a simple baseline method, SUE'', which unlike the other methods considers the freely-available poses of the reference images in the map.
  arXiv  Detail & Related papers  (2024-03-31T03:24:48Z)
• Modeling uncertainty for Gaussian Splatting [21.836830270709]
  We present the first framework for uncertainty estimation using Gaussian Splatting (GS) We introduce a Variational Inference-based approach that seamlessly integrates uncertainty prediction into the common rendering pipeline of GS. We also introduce the Area Under Sparsification Error (AUSE) as a new term in the loss function, enabling optimization of uncertainty estimation alongside image reconstruction.
  arXiv  Detail & Related papers  (2024-03-27T11:45:08Z)
• Instant Uncertainty Calibration of NeRFs Using a Meta-calibrator [60.47106421809998]
  We introduce the concept of a meta-calibrator that performs uncertainty calibration for NeRFs with a single forward pass. We show that the meta-calibrator can generalize on unseen scenes and achieves well-calibrated and state-of-the-art uncertainty for NeRFs.
  arXiv  Detail & Related papers  (2023-12-04T21:29:31Z)
• How to Trust Your Diffusion Model: A Convex Optimization Approach to Conformal Risk Control [9.811982443156063]
  We focus on image-to-image regression tasks and we present a generalization of the Risk-Controlling Prediction Sets (RCPS) procedure. Ours relies on a novel convex optimization approach that allows for multidimensional risk control while provably minimizing the mean interval length. We illustrate our approach on two real-world image denoising problems: on natural images of faces as well as on computed tomography (CT) scans of the abdomen.
  arXiv  Detail & Related papers  (2023-02-07T23:01:16Z)
• Recursively Feasible Probabilistic Safe Online Learning with Control Barrier Functions [60.26921219698514]
  We introduce a model-uncertainty-aware reformulation of CBF-based safety-critical controllers. We then present the pointwise feasibility conditions of the resulting safety controller. We use these conditions to devise an event-triggered online data collection strategy.
  arXiv  Detail & Related papers  (2022-08-23T05:02:09Z)
• Stochastic Attribute Modeling for Face Super-Resolution [23.30144908939099]
  When a high-resolution (HR) image is degraded into a low-resolution (LR) image, the image loses some of the existing information. Most of the existing methods do not consider the uncertainty caused by the attribute, which can only be probabilistically inferred. This paper proposes a novel face super-resolution (SR) scheme to take into the uncertainty by modeling.
  arXiv  Detail & Related papers  (2022-07-16T13:38:05Z)
• Towards Reducing Aleatoric Uncertainty for Medical Imaging Tasks [5.220940151628734]
  Uncertainty in predictions can be attributed to noise or randomness in data (aleatoric) and incorrect model inferences (epistemic) This work proposes a novel approach that interprets data uncertainty estimated from a self-supervised task as noise inherent to the data. Our findings demonstrate the effectiveness of the proposed approach in significantly reducing the aleatoric uncertainty in the image segmentation task.
  arXiv  Detail & Related papers  (2021-10-21T09:31:00Z)
• Uncertainty-aware GAN with Adaptive Loss for Robust MRI Image Enhancement [3.222802562733787]
  Conditional generative adversarial networks (GANs) have shown improved performance in learning photo-realistic image-to-image mappings. This paper proposes a GAN-based framework that (i)models an adaptive loss function for robustness to OOD-noisy data and (ii)estimates the per-voxel uncertainty in the predictions. We demonstrate our method on two key applications in medical imaging: (i)undersampled magnetic resonance imaging (MRI) reconstruction (ii)MRI modality propagation.
  arXiv  Detail & Related papers  (2021-10-07T11:29:03Z)
• PDC-Net+: Enhanced Probabilistic Dense Correspondence Network [161.76275845530964]
  Enhanced Probabilistic Dense Correspondence Network, PDC-Net+, capable of estimating accurate dense correspondences. We develop an architecture and an enhanced training strategy tailored for robust and generalizable uncertainty prediction. Our approach obtains state-of-the-art results on multiple challenging geometric matching and optical flow datasets.
  arXiv  Detail & Related papers  (2021-09-28T17:56:41Z)
• Risk Minimization from Adaptively Collected Data: Guarantees for Supervised and Policy Learning [57.88785630755165]
  Empirical risk minimization (ERM) is the workhorse of machine learning, but its model-agnostic guarantees can fail when we use adaptively collected data. We study a generic importance sampling weighted ERM algorithm for using adaptively collected data to minimize the average of a loss function over a hypothesis class. For policy learning, we provide rate-optimal regret guarantees that close an open gap in the existing literature whenever exploration decays to zero.
  arXiv  Detail & Related papers  (2021-06-03T09:50:13Z)
• Learning Uncertainty For Safety-Oriented Semantic Segmentation In Autonomous Driving [77.39239190539871]
  We show how uncertainty estimation can be leveraged to enable safety critical image segmentation in autonomous driving. We introduce a new uncertainty measure based on disagreeing predictions as measured by a dissimilarity function. We show experimentally that our proposed approach is much less computationally intensive at inference time than competing methods.
  arXiv  Detail & Related papers  (2021-05-28T09:23:05Z)
• Evaluating probabilistic classifiers: Reliability diagrams and score decompositions revisited [68.8204255655161]
  We introduce the CORP approach, which generates provably statistically Consistent, Optimally binned, and Reproducible reliability diagrams in an automated way. Corpor is based on non-parametric isotonic regression and implemented via the Pool-adjacent-violators (PAV) algorithm.
  arXiv  Detail & Related papers  (2020-08-07T08:22:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.