Don't Deceive Me: Mitigating Gaslighting through Attention Reallocation in LMMs

• URL: http://arxiv.org/abs/2504.09456v1
• Date: Sun, 13 Apr 2025 06:47:32 GMT
• Title: Don't Deceive Me: Mitigating Gaslighting through Attention Reallocation in LMMs
• Authors: Pengkun Jiao, Bin Zhu, Jingjing Chen, Chong-Wah Ngo, Yu-Gang Jiang,
• Abstract summary: GasEraser is a training-free approach that reallocates attention weights from misleading textual tokens to semantically salient visual regions.<n>GasEraser significantly improves LMM robustness without requiring retraining or additional supervision.<n>For LLaVA-v1.5-7B, GasEraser reduces the misguidance rate by 48.2%, demonstrating its potential for more trustworthy LMMs.
• Score: 102.18178065928426
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Multimodal Models (LMMs) have demonstrated remarkable capabilities across a wide range of tasks. However, their vulnerability to user gaslighting-the deliberate use of misleading or contradictory inputs-raises critical concerns about their reliability in real-world applications. In this paper, we address the novel and challenging issue of mitigating the negative impact of negation-based gaslighting on LMMs, where deceptive user statements lead to significant drops in model accuracy. Specifically, we introduce GasEraser, a training-free approach that reallocates attention weights from misleading textual tokens to semantically salient visual regions. By suppressing the influence of "attention sink" tokens and enhancing focus on visually grounded cues, GasEraser significantly improves LMM robustness without requiring retraining or additional supervision. Extensive experimental results demonstrate that GasEraser is effective across several leading open-source LMMs on the GaslightingBench. Notably, for LLaVA-v1.5-7B, GasEraser reduces the misguidance rate by 48.2%, demonstrating its potential for more trustworthy LMMs.

Related papers

• Accommodate Knowledge Conflicts in Retrieval-augmented LLMs: Towards Reliable Response Generation in the Wild [11.058848731627233]
  Large language models (LLMs) have advanced information retrieval systems. LLMs often face knowledge conflicts between internal memory and retrievaled external information. We propose Swin-VIB, a novel framework that integrates a pipeline of variational information bottleneck models into adaptive augmentation of retrieved information.
  arXiv  Detail & Related papers  (2025-04-17T14:40:31Z)
• The Other Side of the Coin: Exploring Fairness in Retrieval-Augmented Generation [73.16564415490113]
  Retrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs) by retrieving relevant document from external knowledge sources. We propose two approaches, FairFT and FairFilter, to mitigate the fairness issues introduced by RAG for small-scale LLMs.
  arXiv  Detail & Related papers  (2025-04-11T10:17:10Z)
• Calling a Spade a Heart: Gaslighting Multimodal Large Language Models via Negation [65.92001420372007]
  This paper systematically evaluates state-of-the-art MLLMs across diverse benchmarks. We introduce the first benchmark GaslightingBench, specifically designed to evaluate the vulnerability of MLLMs to negation arguments.
  arXiv  Detail & Related papers  (2025-01-31T10:37:48Z)
• Learning to Correction: Explainable Feedback Generation for Visual Commonsense Reasoning Distractor [12.829202761125096]
  Large multimodal models (LMMs) have shown remarkable performance in the visual commonsense reasoning (VCR) task.<n>However, the ability of LMMs to correct potential visual commonsense errors in the distractor upon their occurrence is yet under-explored.<n>We are the pioneering research for LMMs to simulate this error correction process.
  arXiv  Detail & Related papers  (2024-12-08T03:59:59Z)
• From Yes-Men to Truth-Tellers: Addressing Sycophancy in Large Language Models with Pinpoint Tuning [91.79567270986901]
  Large Language Models (LLMs) tend to prioritize adherence to user prompts over providing veracious responses.<n>Recent works propose to employ supervised fine-tuning (SFT) to mitigate the sycophancy issue.<n>We propose a novel supervised pinpoint tuning (SPT), where the region-of-interest modules are tuned for a given objective.
  arXiv  Detail & Related papers  (2024-09-03T07:01:37Z)
• F-LMM: Grounding Frozen Large Multimodal Models [53.8059045627934]
  We present F-LMM -- grounding frozen off-the-shelf LMMs in human-AI conversations.<n>It is based on the fact that word-pixel correspondences conducive to visual grounding inherently exist in the attention mechanism of well-trained LMMs.<n>It achieves competitive performance on referring expression segmentation and panoptic narrative grounding benchmarks.
  arXiv  Detail & Related papers  (2024-06-09T15:14:26Z)
• Chain-of-Thought Prompting for Demographic Inference with Large Multimodal Models [58.58594658683919]
  Large multimodal models (LMMs) have shown transformative potential across various research tasks. Our findings indicate LMMs possess advantages in zero-shot learning, interpretability, and handling uncurated 'in-the-wild' inputs. We propose a Chain-of-Thought augmented prompting approach, which effectively mitigates the off-target prediction issue.
  arXiv  Detail & Related papers  (2024-05-24T16:26:56Z)
• On Evaluating Adversarial Robustness of Large Vision-Language Models [64.66104342002882]
  We evaluate the robustness of large vision-language models (VLMs) in the most realistic and high-risk setting. In particular, we first craft targeted adversarial examples against pretrained models such as CLIP and BLIP. Black-box queries on these VLMs can further improve the effectiveness of targeted evasion.
  arXiv  Detail & Related papers  (2023-05-26T13:49:44Z)
• Rethinking Uncertainty in Deep Learning: Whether and How it Improves Robustness [20.912492996647888]
  adversarial training (AT) suffers from poor performance both on clean examples and under other types of attacks. Regularizers that encourage uncertain outputs, such as entropy (EntM) and label smoothing (LS) can maintain accuracy on clean examples and improve performance under weak attacks. In this paper, we revisit uncertainty promotion regularizers, including EntM and LS, in the field of adversarial learning.
  arXiv  Detail & Related papers  (2020-11-27T03:22:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.