Beyond Accuracy: On the Effects of Fine-tuning Towards Vision-Language Model's Prediction Rationality

• URL: http://arxiv.org/abs/2412.13333v2
• Date: Mon, 24 Feb 2025 22:07:58 GMT
• Title: Beyond Accuracy: On the Effects of Fine-tuning Towards Vision-Language Model's Prediction Rationality
• Abstract summary: We study the impact of fine-tuning on prediction rationality in vision-language models (VLMs)<n>We propose two new metrics called Prediction Trustworthiness and Inference Reliability.<n>We find that the well-adopted fine-tuning methods led to more correct predictions based on invalid evidence.<n>Having identified valid evidence of target objects, fine-tuned VLMs were more likely to make correct predictions.
• Score: 9.192802430973925
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Vision-Language Models (VLMs), such as CLIP, have already seen widespread applications. Researchers actively engage in further fine-tuning VLMs in safety-critical domains. In these domains, prediction rationality is crucial: the prediction should be correct and based on valid evidence. Yet, for VLMs, the impact of fine-tuning on prediction rationality is seldomly investigated. To study this problem, we proposed two new metrics called Prediction Trustworthiness and Inference Reliability. We conducted extensive experiments on various settings and observed some interesting phenomena. On the one hand, we found that the well-adopted fine-tuning methods led to more correct predictions based on invalid evidence. This potentially undermines the trustworthiness of correct predictions from fine-tuned VLMs. On the other hand, having identified valid evidence of target objects, fine-tuned VLMs were more likely to make correct predictions. Moreover, the findings are also consistent under distributional shifts and across various experimental settings. We hope our research offer fresh insights to VLM fine-tuning.

Related papers

• What LLM Forecasters Know but Don't Say: Probing Internal Representations for Calibration and Faithfulness [3.4789212936038116]
  We train representation-pooling probes on intermediate activations and find they achieve substantially better calibration.<n>We assess chain-of-thought faithfulness through evidence ablation and diversionary injection.<n>A single pre-reasoning pass recovers the committed answer and confidence, and routing questions by the spread of this pre-set answer distribution saves 30-47% of generated tokens.
  arXiv  Detail & Related papers  (2026-07-09T01:52:09Z)
• The Forecast Critic: Leveraging Large Language Models for Poor Forecast Identification [74.64864354503204]
  We propose The Forecast Critic, a system that leverages Large Language Models (LLMs) for automated forecast monitoring.<n>We evaluate the ability of LLMs to assess time series forecast quality.<n>We present three experiments, including on both synthetic and real-world forecasting data.
  arXiv  Detail & Related papers  (2025-12-12T21:59:53Z)
• Domain-Shift-Aware Conformal Prediction for Large Language Models [8.620363085499243]
  We propose a new framework called Domain-Shift-Aware Conformal Prediction (DS-CP)<n>Our framework adapts conformal prediction to large language models under domain shift, by systematically reweighting calibration samples.<n>Our theoretical analysis and experiments on the MMLU benchmark demonstrate that the proposed method delivers more reliable coverage than standard conformal prediction.
  arXiv  Detail & Related papers  (2025-10-07T04:22:06Z)
• Believing without Seeing: Quality Scores for Contextualizing Vision-Language Model Explanations [41.09442370052903]
  We propose evaluating two complementary qualities of VLM-generated explanations via two quality scoring functions.<n>We conduct a user study in which participants have to decide whether a VLM prediction is accurate without viewing its visual context.<n>We observe that showing our quality scores alongside VLM explanations improves participants' accuracy at predicting VLM correctness by 11.1%.
  arXiv  Detail & Related papers  (2025-09-30T06:34:21Z)
• Next-Token Prediction Should be Ambiguity-Sensitive: A Meta-Learning Perspective [12.655285605773932]
  We show that Transformers indeed struggle with high-ambiguity predictions across model sizes.<n>Preliminary results show substantial gains in ambiguous contexts through improved capacity allocation and test-time scalable inference.
  arXiv  Detail & Related papers  (2025-06-19T13:05:12Z)
• TRACE: A Self-Improving Framework for Robot Behavior Forecasting with Vision-Language Models [1.3408365072149797]
  Predicting the near-term behavior of a reactive agent is crucial in many robotic scenarios. We present TRACE, an inference framework that couples tree-of-thought generation with domain-aware feedback. We validate TRACE on both ground-vehicle simulations and real-world marine autonomous surface vehicles.
  arXiv  Detail & Related papers  (2025-03-02T06:58:02Z)
• Uncertainty is Fragile: Manipulating Uncertainty in Large Language Models [79.76293901420146]
  Large Language Models (LLMs) are employed across various high-stakes domains, where the reliability of their outputs is crucial. Our research investigates the fragility of uncertainty estimation and explores potential attacks. We demonstrate that an attacker can embed a backdoor in LLMs, which, when activated by a specific trigger in the input, manipulates the model's uncertainty without affecting the final output.
  arXiv  Detail & Related papers  (2024-07-15T23:41:11Z)
• Coverage-Guaranteed Prediction Sets for Out-of-Distribution Data [11.416180794737203]
  Splital prediction ( SCP) is an efficient framework for handling the confidence set prediction problem. We show that trivially applying SCP results in a failure to maintain the marginal coverage when the unseen target domain is different from the source domain. We develop a method for forming confident prediction sets in the OOD setting and theoretically prove the validity of our method.
  arXiv  Detail & Related papers  (2024-03-29T03:16:29Z)
• Selective "Selective Prediction": Reducing Unnecessary Abstention in Vision-Language Reasoning [67.82016092549284]
  We introduce ReCoVERR, an inference-time algorithm to reduce the over-abstention of a selective vision-language system. ReCoVERR tries to find relevant clues in an image that provide additional evidence for the prediction.
  arXiv  Detail & Related papers  (2024-02-23T21:16:52Z)
• Source-Free Unsupervised Domain Adaptation with Hypothesis Consolidation of Prediction Rationale [53.152460508207184]
  Source-Free Unsupervised Domain Adaptation (SFUDA) is a challenging task where a model needs to be adapted to a new domain without access to target domain labels or source domain data. This paper proposes a novel approach that considers multiple prediction hypotheses for each sample and investigates the rationale behind each hypothesis. To achieve the optimal performance, we propose a three-step adaptation process: model pre-adaptation, hypothesis consolidation, and semi-supervised learning.
  arXiv  Detail & Related papers  (2024-02-02T05:53:22Z)
• Performative Time-Series Forecasting [71.18553214204978]
  We formalize performative time-series forecasting (PeTS) from a machine-learning perspective. We propose a novel approach, Feature Performative-Shifting (FPS), which leverages the concept of delayed response to anticipate distribution shifts. We conduct comprehensive experiments using multiple time-series models on COVID-19 and traffic forecasting tasks.
  arXiv  Detail & Related papers  (2023-10-09T18:34:29Z)
• Multiclass Alignment of Confidence and Certainty for Network Calibration [10.15706847741555]
  Recent studies reveal that deep neural networks (DNNs) are prone to making overconfident predictions. We propose a new train-time calibration method, which features a simple, plug-and-play auxiliary loss known as multi-class alignment of predictive mean confidence and predictive certainty (MACC) Our method achieves state-of-the-art calibration performance for both in-domain and out-domain predictions.
  arXiv  Detail & Related papers  (2023-09-06T00:56:24Z)
• VLUCI: Variational Learning of Unobserved Confounders for Counterfactual Inference [11.191748173380539]
  Causal inference plays a vital role in diverse domains like epidemiology, healthcare, and economics. De-confounding and counterfactual prediction in observational data has emerged as a prominent concern in causal inference research. We propose a novel variational learning model of unobserved confounders for counterfactual inference.
  arXiv  Detail & Related papers  (2023-08-02T01:44:30Z)
• Confidence and Dispersity Speak: Characterising Prediction Matrix for Unsupervised Accuracy Estimation [51.809741427975105]
  This work aims to assess how well a model performs under distribution shifts without using labels. We use the nuclear norm that has been shown to be effective in characterizing both properties. We show that the nuclear norm is more accurate and robust in accuracy than existing methods.
  arXiv  Detail & Related papers  (2023-02-02T13:30:48Z)
• VisFIS: Visual Feature Importance Supervision with Right-for-the-Right-Reason Objectives [84.48039784446166]
  We show that model FI supervision can meaningfully improve VQA model accuracy as well as performance on several Right-for-the-Right-Reason metrics. Our best performing method, Visual Feature Importance Supervision (VisFIS), outperforms strong baselines on benchmark VQA datasets. Predictions are more accurate when explanations are plausible and faithful, and not when they are plausible but not faithful.
  arXiv  Detail & Related papers  (2022-06-22T17:02:01Z)
• Multi-agent Performative Prediction: From Global Stability and Optimality to Chaos [42.40985526691935]
  We introduce a natural multi-agent version of this framework, where multiple decision makers try to predict the same outcome. We showcase that such competition can result in interesting phenomena by proving the possibility of phase transitions from stability to instability and eventually chaos.
  arXiv  Detail & Related papers  (2022-01-25T17:26:12Z)
• Learning to Predict Trustworthiness with Steep Slope Loss [69.40817968905495]
  We study the problem of predicting trustworthiness on real-world large-scale datasets. We observe that the trustworthiness predictors trained with prior-art loss functions are prone to view both correct predictions and incorrect predictions to be trustworthy. We propose a novel steep slope loss to separate the features w.r.t. correct predictions from the ones w.r.t. incorrect predictions by two slide-like curves that oppose each other.
  arXiv  Detail & Related papers  (2021-09-30T19:19:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.