CARGO-VL: Counterfactual Arbitration with Risk-Constrained Group Optimization for Vision-Language Models

• URL: http://arxiv.org/abs/2608.04509v1
• Date: Wed, 05 Aug 2026 06:47:46 GMT
• Title: CARGO-VL: Counterfactual Arbitration with Risk-Constrained Group Optimization for Vision-Language Models
• Abstract summary: Vision-language systems combine images with retrieved text, but these sources can disagree or jointly fail to support an answer.<n>We introduce CARGO-VL, a group-relative framework that optimize matched variants covering aligned, image-correct, text-correct, and both-wrong (A/V/T/N) evidence states.<n>We also contribute XMC (eXtended Modal Conflict), a four-condition conflict training resource, and evaluate transfer on CMC-Bench and Modality-Bias.
• Score: 2.038560872025072
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-language systems combine images with retrieved text, but these sources can disagree or jointly fail to support an answer. Reliable models must identify the trustworthy source and abstain when neither is adequate. Existing post-training objectives score instances independently and therefore do not enforce coherent behavior under counterfactual evidence changes. We introduce CARGO-VL, a group-relative framework that optimizes matched variants covering aligned, image-correct, text-correct, and both-wrong (A/V/T/N) evidence states as one bundle. Its objective couples condition-wise correctness with transition rewards for answer invariance, source equivariance, and answer-to-abstention switching, while a primal-dual controller balances unsafe answers against excessive deferral. We also contribute XMC (eXtended Modal Conflict), a four-condition conflict training resource, and evaluate transfer on CMC-Bench and Modality-Bias. Across multiple seeds, CARGO-VL improves conflict handling, unsupported-answer avoidance, and modality balance over pointwise baselines. Ablations identify complementary benefits from relational transition signals and adaptive risk control, supporting counterfactual consistency as a practical objective for reliable multimodal evidence arbitration.

Related papers

• CONFER: Conflict-Aware Evidence Negotiation for Regime-Calibrated Weak Supervision in Multimodal Emotion Recognition [2.587219755021037]
  graph-based conflict-aware evidence negotiation framework for weakly supervised multimodal emotion recognition.<n>ConFER represents each modality expert as a node with a predictive belief, boundary-based uncertainty, and runtime reliability estimated from historical out-of-fold performance and current-sample uncertainty.<n>ConFER achieves competitive performance, reaching textbf0.873 accuracy on AMIGOS-V and textbf0.854 accuracy on MAHNOB-V under strict LOSO evaluation.
  arXiv  Detail & Related papers  (2026-08-08T02:42:00Z)
• COSMO: Consensus-Driven Shift Modulation for Source-Free Domain Adaptation [79.43842208099619]
  Source-free domain adaptation (SFDA) adapts a source-trained model to an unlabeled target domain without source data.<n>We formulate VLM-guided SFDA as a sample-wise reliability-allocation problem.<n>We propose Consensus-Driven Shift Modulation (COSMO) to replace expert-to-expert guidance with co-adaptation.
  arXiv  Detail & Related papers  (2026-08-05T09:10:30Z)
• SVR: Self-Verifying Refinement via Joint Verdict-Confidence Reinforcement Learning for Adaptive Test-Time Compute [62.3458279176813]
  Self-Verifying Refinement is an oracle-free multi-turn reinforcement learning framework.<n>It learns to use self-verification as a compute-control policy.<n>It achieves a macro-average accuracy of 0.563 with only 2.99 inference turns on average.
  arXiv  Detail & Related papers  (2026-07-30T16:20:58Z)
• Visual Credit Audit for Multimodal Spatial Reasoning [70.16915309526443]
  Visual Credit Audit separates two Estimands: whether the benchmark image gives the model's declared decision more support than text-only and blank controls, and whether the model responds to relation-specific visual evidence.<n>Applying labels yields dependence-credited correctness (D-CC)<n>Across four open MLLMs and two spatial benchmarks, 12.73-26.25% of decisions are correct yet uncredited.
  arXiv  Detail & Related papers  (2026-07-29T15:55:31Z)
• TwistedMerge: Certified Higher-Order Diagnostics and Abstention for Model Merging [0.48086260459837454]
  We formulate merging as a finite descent problem in which checkpoints are local objects, alignment maps are transitions, and cycle products are residuals.<n>We prove a constant-edge no-go result, frozen-complex three-way and pre-family error-control theorems, and a refinement test for comparison-complex sensitivity.<n>A trained low-rank-adapter audit shows that naive factor averaging depends on the chosen GLr representative, whereas global factor synchronization and dense-delta SVD are stable.
  arXiv  Detail & Related papers  (2026-07-23T03:24:50Z)
• Trust but Verify: Prover-Verifier Deliberation for Selective LLM Prediction [4.273094752480624]
  We introduce prover-verifier deliberation (PVD), an inference-time protocol grounded in interactive proof theory.<n>PVD produces both an answer and a structured confidence verdict, allowing a system to report high-confidence answers while abstaining on uncertain cases.
  arXiv  Detail & Related papers  (2026-05-24T15:23:27Z)
• To Fuse or to Drop? Dual-Path Learning for Resolving Modality Conflicts in Multimodal Emotion Recognition [87.22988227382329]
  Multimodal emotion recognition (MER) benefits from combining text, audio, and vision, yet standard fusion often fails when modalities conflict.<n>We propose Dual-Path Conflict Resolution (DCR), a unified framework that learns when to fuse and when to drop modalities.
  arXiv  Detail & Related papers  (2026-05-06T13:11:33Z)
• Multimodal Learning on Low-Quality Data with Conformal Predictive Self-Calibration [72.0672328514289]
  Multimodal learning often grapples with the challenge of low-quality data.<n>We propose a unified framework, termed Conformal Predictive Self-Calibration.<n>Our framework consistently outperforms existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-05-05T14:48:52Z)
• FACT-E: Causality-Inspired Evaluation for Trustworthy Chain-of-Thought Reasoning [49.65751420291115]
  Chain-of-Thought (CoT) prompting has improved LLM reasoning, but models often generate explanations that appear coherent while containing unfaithful intermediate steps.<n>We propose FACT-E, a causality-inspired framework for evaluating CoT quality.<n>We show that FACT-E improves reasoning-trajectory selection and yields stronger in-context learnings.
  arXiv  Detail & Related papers  (2026-04-12T15:35:08Z)
• Trusted Uncertainty in Large Language Models: A Unified Framework for Confidence Calibration and Risk-Controlled Refusal [31.458406135473805]
  We present UniCR, a unified framework that turns heterogeneous uncertainty evidence into a calibrated probability of correctness.<n>UniCR learns a lightweight calibration head with temperature scaling and proper scoring.<n>Experiments on short-form QA, code generation with execution tests, and retrieval-augmented long-form QA show consistent improvements in calibration metrics.
  arXiv  Detail & Related papers  (2025-09-01T13:14:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.