RePair: Turning Retrieval Failures into Counterfactual Hard Pairs

• URL: http://arxiv.org/abs/2608.29604v1
• Date: Sun, 30 Aug 2026 06:47:38 GMT
• Title: RePair: Turning Retrieval Failures into Counterfactual Hard Pairs
• Abstract summary: RePair mines false positives bidirectionally, applies LLM-guided counterfactual editing, and trains with a local hard-pair contrastive objective.<n>On Flickr30K and COCO30K, RePair outperforms controlled augmentation baselines with only 107K synthetic samples.
• Score: 15.148342372814662
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision-language retrieval with CLIP-style dual encoders achieves strong cross-modal performance, yet practical accuracy often hinges on localized semantic distinctions where top-ranked near misses differ from the true match by a single critical detail. Hard-sample mining can select confusable candidates but cannot construct corrected counterparts; synthetic augmentation can generate novel samples but, without conditioning on actual model failures, targets irrelevant dimensions of hardness. We observe that a top-ranked false positive is a counterfactual scaffold---sharing most of the query's semantics while differing in a localized failure-causing residual. Minimally correcting this residual yields a hard positive of the ground truth in the same modality; the corrected and unedited versions form a hard negative pair that straddles the decision boundary, producing complementary pull--push supervision. We introduce RePair, guided by three principles---Validity, Minimality, and Locality---which mines false positives bidirectionally, applies LLM-guided counterfactual editing, and trains with a local hard-pair contrastive objective. On Flickr30K and COCO30K, RePair outperforms controlled augmentation baselines with only 107K synthetic samples---26\%--75\% fewer than comparable methods---confirming failure-conditioned repair is more data-efficient than error-agnostic augmentation.

Related papers

• Wrong Prediction, Right Answer: Recovering Evidence from Collapsed LLM Sequence Scores [31.709549159768727]
  We observe a consistent readout gap across diverse reasoning benchmarks.<n> hidden-state probes successfully decode correct answers even when native sequence scoring completely collapses.<n>Our results show that many apparent zero-shot reasoning deficits are expression failures masking intact internal logic.
  arXiv  Detail & Related papers  (2026-08-31T16:43:55Z)
• When Does Consensus Mean Correctness? Measuring the Agreement-Accuracy Coupling with Semantics-Preserving Re-Rendering [0.8594140167290097]
  A model's agreement across perturbed inputs is used as a label-free reliability signal.<n>Rerendering beats resampling on both accuracy and reliability.<n>Fine-tuning on the model's own cross-render consensus inverts.
  arXiv  Detail & Related papers  (2026-08-06T07:08:05Z)
• Compositional Multi-hop Factual Error Correction via Decomposition-and-Injection [60.34509125517578]
  We present CECoR (Compositional Error Correction via Reasoning-aware Synthesis), a reasoning-aware framework for compositional error correction.<n> CECoR decomposes multi-hop claims into interpretable reasoning steps and injects controlled perturbations to synthesize high-quality training pairs.<n>A two-stage learning strategy combining supervised fine-tuning and reinforcement learning improves factual accuracy and robustness.
  arXiv  Detail & Related papers  (2026-05-04T07:08:39Z)
• CARE What Fails: Contrastive Anchored-REflection for Verifiable Multimodal [84.71254539482369]
  Group-relative reinforcement learning with verifiable rewards (RLVR) often wastes the most informative data it already has the failures.<n>We present CARE, a failure-centric post-training framework for multimodal reasoning that turns errors into supervision.<n> CARE improves accuracy and training smoothness while explicitly increasing the share of learning signal that comes from failures.
  arXiv  Detail & Related papers  (2025-12-22T16:34:21Z)
• What Does It Take to Build a Performant Selective Classifier? [30.90225954725644]
  Bayes noise, approximation error, ranking error, statistical noise, and implementation- or shift-induced slack are studied.<n>We validate our decomposition on synthetic two-moons data and on real-world vision and language benchmarks.<n>Our results confirm that (i) Bayes noise and limited model capacity can account for substantial gaps, (ii) only richer, feature-aware calibrators meaningfully improve score ordering, and (iii) data shift introduces a separate slack that demands distributionally robust training.
  arXiv  Detail & Related papers  (2025-10-23T05:48:40Z)
• Sample Smart, Not Hard: Correctness-First Decoding for Better Reasoning in LLMs [72.82403830490084]
  We argue that the decoding rule should be calibrated by correctness, not confidence alone.<n>We propose simple strategies that achieve this goal: Greedy-Threshold makes sampling greedy at very low confidence steps.<n>Together, our findings challenge prevailings about decoding under uncertainty and show gains across math and general reasoning benchmarks.
  arXiv  Detail & Related papers  (2025-10-07T14:46:12Z)
• Latent Veracity Inference for Identifying Errors in Stepwise Reasoning [78.29317733206643]
  We introduce Veracity Search (VS), a discrete search algorithm over veracity assignments.<n>It performs otherwise intractable inference in the posterior distribution over latent veracity values.<n>It generalizes VS, enabling accurate zero-shot veracity inference in novel contexts.
  arXiv  Detail & Related papers  (2025-05-17T04:16:36Z)
• TrustLoRA: Low-Rank Adaptation for Failure Detection under Out-of-distribution Data [62.22804234013273]
  We propose a simple failure detection framework to unify and facilitate classification with rejection under both covariate and semantic shifts.<n>Our key insight is that by separating and consolidating failure-specific reliability knowledge with low-rank adapters, we can enhance the failure detection ability effectively and flexibly.
  arXiv  Detail & Related papers  (2025-04-20T09:20:55Z)
• Noisy Correspondence Learning with Self-Reinforcing Errors Mitigation [63.180725016463974]
  Cross-modal retrieval relies on well-matched large-scale datasets that are laborious in practice. We introduce a novel noisy correspondence learning framework, namely textbfSelf-textbfReinforcing textbfErrors textbfMitigation (SREM)
  arXiv  Detail & Related papers  (2023-12-27T09:03:43Z)
• Fairness under Covariate Shift: Improving Fairness-Accuracy tradeoff with few Unlabeled Test Samples [21.144077993862652]
  We operate in the unsupervised regime where only a small set of unlabeled test samples along with a labeled training set is available. We experimentally verify that optimizing with our loss formulation outperforms a number of state-of-the-art baselines. We show that our proposed method significantly outperforms them.
  arXiv  Detail & Related papers  (2023-10-11T14:39:51Z)
• Robustness and Accuracy Could Be Reconcilable by (Proper) Definition [109.62614226793833]
  The trade-off between robustness and accuracy has been widely studied in the adversarial literature. We find that it may stem from the improperly defined robust error, which imposes an inductive bias of local invariance. By definition, SCORE facilitates the reconciliation between robustness and accuracy, while still handling the worst-case uncertainty.
  arXiv  Detail & Related papers  (2022-02-21T10:36:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.