BRUCE: Benchmarking Robustness Under Corruption Escalation for Scientific Vision-Language Reasoning

• URL: http://arxiv.org/abs/2608.07742v1
• Date: Fri, 07 Aug 2026 20:19:11 GMT
• Title: BRUCE: Benchmarking Robustness Under Corruption Escalation for Scientific Vision-Language Reasoning
• Abstract summary: We propose BRUCE (Benchmarking Robustness Under Corruption Escalation), a framework for scientific vision-language reasoning.<n>BRUCE employs two novel metrics -- Robustness Corruption Index (RCI) and Traversal-RCI (T-RCI) -- to quantify how rapidly multimodal reasoning performance deteriorates.<n>We evaluate BRUCE across chemistry and mathematical reasoning tasks for multiple datasets, while analyzing corruption-induced prediction failures.
• Score: 2.717526933594264
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Visual-language models (VLMs) frequently struggle with robustness issues in real-world situations due to low- or varying-quality input images. In this paper, we aim at analyzing VLMs' robustness by applying perturbations and distortions to the input images, such as blur or low contrast. Toward this goal, we propose BRUCE (Benchmarking Robustness Under Corruption Escalation, a multimodal reasoning fragility framework for scientific vision-language reasoning. State-of-the-art evaluation frameworks/studies primarily focus on clean-task accuracy and rarely analyze how reasoning stability degrades across robustness dimensions. Besides varying over a wide-range of input perturbations, BRUCE employs two novel metrics -- Robustness Corruption Index (RCI) and Traversal-RCI (T-RCI) -- to quantify how rapidly multimodal reasoning performance deteriorates in VLMs as visual corruption severity increases under progressive perturbation scaling. We evaluate BRUCE across chemistry and mathematical reasoning tasks for multiple datasets, while analyzing corruption-induced prediction failures in terms of four high-level reasoning domains: OCR-dependent reasoning, spatial reasoning, symbolic reasoning, and semantic failures, with each containing fine-grained corruption specific failure subtypes, thereby enabling an interpretable failure analysis.

Related papers

• Evidence-Grounded Trustworthy Multimodal Reasoning and Evaluation Benchmark in Complex Urban Scenes [37.743333035442426]
  This paper introduces AD2-Bench, which decomposes reasoning into a structured Chain of Evidence (CoE)<n>Building on this perspective, the authors identify two primary causes of reasoning failure: Spatial Ambiguity and Semantic Uncertainty.<n>They then propose Evidence-grounded Visual Reasoning (EGVOR), which replaces implicit reasoning with the explicit generation of Evidence Atoms.
  arXiv  Detail & Related papers  (2026-08-11T14:23:07Z)
• The Cost of Context: Mitigating Textual Bias in Multimodal Retrieval-Augmented Generation [18.83395586542378]
  Multimodal Large Language Models (MLLMs) are increasingly integrated with Retrieval-Augmented Generation (RAG)<n>We identify and formalize the phenomenon of recorruption, where the introduction of even perfectly accurate "oracle" context causes a capable model to abandon an initially correct prediction.<n>We propose Bottleneck Attention Intervention for Recovery (BAIR), a parameter-free, inference-time framework that restores visual saliency and applies position-aware penalties to textual distractors.
  arXiv  Detail & Related papers  (2026-05-07T02:27:04Z)
• METER: Evaluating Multi-Level Contextual Causal Reasoning in Large Language Models [61.33372454250959]
  Contextual causal reasoning is a critical yet challenging capability for Large Language Models.<n>Existing benchmarks fail to ensure context consistency or cover the full causal hierarchy.<n>We pioneer METER to systematically benchmark LLMs across all three levels of the causal ladder.
  arXiv  Detail & Related papers  (2026-04-13T14:07:11Z)
• Robust-R1: Degradation-Aware Reasoning for Robust Visual Understanding [54.05243949024302]
  Existing robust MLLMs rely on implicit training/adaptation that focuses solely on visual encoder generalization.<n>We propose Robust-R1, a novel framework that explicitly models visual degradations through structured reasoning chains.<n>Our approach integrates: (i) supervised fine-tuning for degradation-aware reasoning foundations, (ii) reward-driven alignment for accurately perceiving degradation parameters, and (iii) dynamic reasoning depth scaling adapted to degradation intensity.
  arXiv  Detail & Related papers  (2025-12-19T12:56:17Z)
• Benchmarking Corruption Robustness of LVLMs: A Discriminative Benchmark and Robustness Alignment Metric [49.393713730706445]
  We introduce Bench-C, a benchmark emphasizing discriminative samples for assessing corruption robustness.<n>We propose the Robustness Alignment Score (RAS), a unified metric that measures degradation in logit-level prediction structure.
  arXiv  Detail & Related papers  (2025-11-24T12:07:56Z)
• LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Models [49.92148175114169]
  We perform a systematic vulnerability analysis by introducing controlled perturbations across seven dimensions.<n>Models exhibit extreme sensitivity to perturbation factors, including camera viewpoints and robot initial states.<n>Surprisingly, models are largely insensitive to language variations, with further experiments revealing that models tend to ignore language instructions completely.
  arXiv  Detail & Related papers  (2025-10-15T14:51:36Z)
• Rethinking Evaluation of Infrared Small Target Detection [105.59753496831739]
  This paper introduces a hybrid-level metric incorporating pixel- and target-level performance, proposing a systematic error analysis method, and emphasizing the importance of cross-dataset evaluation.<n>An open-source toolkit has be released to facilitate standardized benchmarking.
  arXiv  Detail & Related papers  (2025-09-21T02:45:07Z)
• Analysing the Robustness of Vision-Language-Models to Common Corruptions [2.9459935333120972]
  Vision-language models (VLMs) have demonstrated impressive capabilities in understanding and reasoning about visual and textual content.<n>We present the first comprehensive analysis of VLM robustness across 19 corruption types from the ImageNet-C benchmark.<n>We introduce two new benchmarks, TextVQA-C and GQA-C, to evaluate how corruptions affect scene text understanding and object-based reasoning.
  arXiv  Detail & Related papers  (2025-04-18T13:46:32Z)
• Benchmarking the Spatial Robustness of DNNs via Natural and Adversarial Localized Corruptions [49.546479320670464]
  This paper introduces specialized metrics for benchmarking the spatial robustness of segmentation models.<n>We propose region-aware multi-attack adversarial analysis, a method that enables a deeper understanding of model robustness.<n>The results reveal that models respond to these two types of threats differently.
  arXiv  Detail & Related papers  (2025-04-02T11:37:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.