On the robustness of multimodal language model towards distractions

• URL: http://arxiv.org/abs/2502.09818v1
• Date: Thu, 13 Feb 2025 23:29:01 GMT
• Title: On the robustness of multimodal language model towards distractions
• Authors: Ming Liu, Hao Chen, Jindong Wang, Wensheng Zhang,
• Abstract summary: This paper aims to assess the robustness of vision-language models (VLMs) against both visual and textual distractions in the context of science question answering. Our findings reveal that most-of-the-art VLMs, including GPT-4, are vulnerable to various types of distractions, experiencing noticeable degradation in reasoning capabilities when confronted with distractions.
• Score: 16.144509808157643
• License:
• Abstract: Although vision-language models (VLMs) have achieved significant success in various applications such as visual question answering, their resilience to prompt variations remains an under-explored area. Understanding how distractions affect VLMs is crucial for improving their real-world applicability, as inputs could have noisy and irrelevant information in many practical scenarios. This paper aims to assess the robustness of VLMs against both visual and textual distractions in the context of science question answering. Built on the ScienceQA dataset, we developed a new benchmark that introduces distractions in both the visual and textual contexts to evaluate the reasoning capacity of VLMs amid these distractions. Our findings reveal that most-of-the-art VLMs, including GPT-4, are vulnerable to various types of distractions, experiencing noticeable degradation in reasoning capabilities when confronted with distractions. Notably, models such as InternVL2 demonstrate a higher degree of robustness to these distractions. We also found that models exhibit greater sensitivity to textual distractions than visual ones. Additionally, we explored various mitigation strategies, such as prompt engineering, to counteract the impact of distractions. While these strategies improved solution accuracy, our analysis shows that there remain significant opportunities for improvement.

Related papers

• When Data Manipulation Meets Attack Goals: An In-depth Survey of Attacks for VLMs [15.74045364570382]
  We present an in-depth survey of the attack strategies tailored for Vision-Language Models (VLMs) We categorize these attacks based on their underlying objectives. We outline corresponding defense mechanisms that have been proposed to mitigate these vulnerabilities.
  arXiv  Detail & Related papers  (2025-02-10T12:20:08Z)
• Evaluating Vision-Language Models for Emotion Recognition [1.7409710986849658]
  We present the first comprehensive evaluation of Large Vision-Language Models (VLMs) for recognizing evoked emotions from images. Through several experiments, we demonstrate important factors that emotion recognition performance depends on, and also characterize the various errors made by VLMs in the process.
  arXiv  Detail & Related papers  (2025-02-08T18:25:31Z)
• Breaking Focus: Contextual Distraction Curse in Large Language Models [68.4534308805202]
  We investigate a critical vulnerability in Large Language Models (LLMs) This phenomenon arises when models fail to maintain consistent performance on questions modified with semantically coherent but irrelevant context. We propose an efficient tree-based search methodology to automatically generate CDV examples.
  arXiv  Detail & Related papers  (2025-02-03T18:43:36Z)
• DRIVINGVQA: Analyzing Visual Chain-of-Thought Reasoning of Vision Language Models in Real-World Scenarios with Driving Theory Tests [69.00444996464662]
  We present DrivingVQA, a new benchmark derived from driving theory tests to evaluate visual chain-of-thought reasoning in complex real-world scenarios. Our experiments reveal that open-source and proprietary LVLMs struggle with visual chain-of-thought reasoning under zero-shot settings. We investigate training strategies that leverage relevant entities to improve visual reasoning.
  arXiv  Detail & Related papers  (2025-01-08T18:31:16Z)
• Insight Over Sight? Exploring the Vision-Knowledge Conflicts in Multimodal LLMs [55.74117540987519]
  This paper explores the problem of commonsense-level vision-knowledge conflict in Multimodal Large Language Models (MLLMs) We introduce an automated pipeline, augmented with human-in-the-loop quality control, to establish a benchmark aimed at simulating and assessing the conflicts in MLLMs. We evaluate the conflict-resolution capabilities of nine representative MLLMs across various model families and find a noticeable over-reliance on textual queries.
  arXiv  Detail & Related papers  (2024-10-10T17:31:17Z)
• MarvelOVD: Marrying Object Recognition and Vision-Language Models for Robust Open-Vocabulary Object Detection [107.15164718585666]
  We investigate the root cause of VLMs' biased prediction under the open vocabulary detection context. Our observations lead to a simple yet effective paradigm, coded MarvelOVD, that generates significantly better training targets. Our method outperforms the other state-of-the-arts by significant margins.
  arXiv  Detail & Related papers  (2024-07-31T09:23:57Z)
• RUPBench: Benchmarking Reasoning Under Perturbations for Robustness Evaluation in Large Language Models [12.112914393948415]
  We present RUPBench, a benchmark designed to evaluate large language models (LLMs) across diverse reasoning tasks. Our benchmark incorporates 15 reasoning datasets, categorized into commonsense, arithmetic, logical, and knowledge-intensive reasoning. By examining the performance of state-of-the-art LLMs such as GPT-4o, Llama3, Phi-3, and Gemma on both original and perturbed datasets, we provide a detailed analysis of their robustness and error patterns.
  arXiv  Detail & Related papers  (2024-06-16T17:26:44Z)
• Evaluating and Improving Continual Learning in Spoken Language Understanding [58.723320551761525]
  We propose an evaluation methodology that provides a unified evaluation on stability, plasticity, and generalizability in continual learning. By employing the proposed metric, we demonstrate how introducing various knowledge distillations can improve different aspects of these three properties of the SLU model.
  arXiv  Detail & Related papers  (2024-02-16T03:30:27Z)
• Evaluation and Enhancement of Semantic Grounding in Large Vision-Language Models [25.413601452403213]
  Large Vision-Language Models (LVLMs) offer remarkable benefits for a variety of vision-language tasks. Their constrained semantic grounding ability hinders their application in real-world scenarios. We propose a data-centric enhancement method that aims to improve LVLMs' semantic grounding ability.
  arXiv  Detail & Related papers  (2023-09-07T22:59:56Z)
• Visual Perturbation-aware Collaborative Learning for Overcoming the Language Prior Problem [60.0878532426877]
  We propose a novel collaborative learning scheme from the viewpoint of visual perturbation calibration. Specifically, we devise a visual controller to construct two sorts of curated images with different perturbation extents. The experimental results on two diagnostic VQA-CP benchmark datasets evidently demonstrate its effectiveness.
  arXiv  Detail & Related papers  (2022-07-24T23:50:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.