Phi: Preference Hijacking in Multi-modal Large Language Models at Inference Time

• URL: http://arxiv.org/abs/2509.12521v1
• Date: Mon, 15 Sep 2025 23:55:57 GMT
• Title: Phi: Preference Hijacking in Multi-modal Large Language Models at Inference Time
• Authors: Yifan Lan, Yuanpu Cao, Weitong Zhang, Lu Lin, Jinghui Chen,
• Abstract summary: We introduce a novel method for manipulating the MLLM response preferences using a preference hijacked image.<n>Our method works at inference time and requires no model modifications.<n> Experimental results across various tasks demonstrate the effectiveness of our approach.
• Score: 39.97820478987012
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recently, Multimodal Large Language Models (MLLMs) have gained significant attention across various domains. However, their widespread adoption has also raised serious safety concerns. In this paper, we uncover a new safety risk of MLLMs: the output preference of MLLMs can be arbitrarily manipulated by carefully optimized images. Such attacks often generate contextually relevant yet biased responses that are neither overtly harmful nor unethical, making them difficult to detect. Specifically, we introduce a novel method, Preference Hijacking (Phi), for manipulating the MLLM response preferences using a preference hijacked image. Our method works at inference time and requires no model modifications. Additionally, we introduce a universal hijacking perturbation -- a transferable component that can be embedded into different images to hijack MLLM responses toward any attacker-specified preferences. Experimental results across various tasks demonstrate the effectiveness of our approach. The code for Phi is accessible at https://github.com/Yifan-Lan/Phi.

Related papers

• Odysseus: Jailbreaking Commercial Multimodal LLM-integrated Systems via Dual Steganography [77.44136793431893]
  We propose a novel jailbreak paradigm that introduces dual steganography to covertly embed malicious queries into benign-looking images.<n>Our Odysseus successfully jailbreaks several pioneering and realistic MLLM-integrated systems, achieving up to 99% attack success rate.
  arXiv  Detail & Related papers  (2025-12-23T08:53:36Z)
• Adversarial Confusion Attack: Disrupting Multimodal Large Language Models [1.4037095606573826]
  We introduce the Adversarial Confusion Attack, a new class of threats against multimodal large language models (MLLMs)<n>Unlike jailbreaks or targeted misclassification, the goal is to induce systematic disruption that makes the model generate incoherent or confidently incorrect outputs.<n> Practical applications include embedding such adversarial images into websites to prevent MLLM-powered AI Agents from operating reliably.
  arXiv  Detail & Related papers  (2025-11-25T17:00:31Z)
• Friend or Foe: How LLMs' Safety Mind Gets Fooled by Intent Shift Attack [53.34204977366491]
  Large language models (LLMs) remain vulnerable to jailbreaking attacks despite their impressive capabilities.<n>In this paper, we introduce ISA (Intent Shift Attack), which obfuscates LLMs about the intent of the attacks.<n>Our approach only needs minimal edits to the original request, and yields natural, human-readable, and seemingly harmless prompts.
  arXiv  Detail & Related papers  (2025-11-01T13:44:42Z)
• JPS: Jailbreak Multimodal Large Language Models with Collaborative Visual Perturbation and Textual Steering [73.962469626788]
  Jailbreak attacks against multimodal large language Models (MLLMs) are a significant research focus.<n>We propose JPS, underlineJailbreak MLLMs with collaborative visual underlinePerturbation and textual underlineSteering.
  arXiv  Detail & Related papers  (2025-08-07T07:14:01Z)
• Jailbreaking Multimodal Large Language Models via Shuffle Inconsistency [26.320250214125483]
  Multimodal Large Language Models (MLLMs) have achieved impressive performance and have been put into practical use in commercial applications.<n>Jailbreak attacks aim to bypass safety mechanisms and discover MLLMs' potential risks.<n>We propose a text-image jailbreak attack named SI-Attack to overcome the Shuffle Inconsistency and overcome the shuffle randomness.
  arXiv  Detail & Related papers  (2025-01-09T02:47:01Z)
• CoCA: Regaining Safety-awareness of Multimodal Large Language Models with Constitutional Calibration [90.36429361299807]
  multimodal large language models (MLLMs) have demonstrated remarkable success in engaging in conversations involving visual inputs. The integration of visual modality has introduced a unique vulnerability: the MLLM becomes susceptible to malicious visual inputs. We introduce a technique termed CoCA, which amplifies the safety-awareness of the MLLM by calibrating its output distribution.
  arXiv  Detail & Related papers  (2024-09-17T17:14:41Z)
• Human-Interpretable Adversarial Prompt Attack on Large Language Models with Situational Context [49.13497493053742]
  This research explores converting a nonsensical suffix attack into a sensible prompt via a situation-driven contextual re-writing. We combine an independent, meaningful adversarial insertion and situations derived from movies to check if this can trick an LLM. Our approach demonstrates that a successful situation-driven attack can be executed on both open-source and proprietary LLMs.
  arXiv  Detail & Related papers  (2024-07-19T19:47:26Z)
• Refusing Safe Prompts for Multi-modal Large Language Models [36.276781604895454]
  We introduce MLLM-Refusal, the first method that induces refusals for safe prompts. We formulate MLLM-Refusal as a constrained optimization problem and propose an algorithm to solve it. We evaluate MLLM-Refusal on four MLLMs across four datasets.
  arXiv  Detail & Related papers  (2024-07-12T07:18:05Z)
• Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Models [107.88745040504887]
  We study the harmlessness alignment problem of multimodal large language models (MLLMs)<n>Inspired by this, we propose a novel jailbreak method named HADES, which hides and amplifies the harmfulness of the malicious intent within the text input.<n> Experimental results show that HADES can effectively jailbreak existing MLLMs, which achieves an average Attack Success Rate (ASR) of 90.26% for LLaVA-1.5 and 71.60% for Gemini Pro Vision.
  arXiv  Detail & Related papers  (2024-03-14T18:24:55Z)
• MLLM-Protector: Ensuring MLLM's Safety without Hurting Performance [36.03512474289962]
  This paper investigates the novel challenge of defending MLLMs against malicious attacks through visual inputs. Images act as a foreign language" that is not considered during safety alignment, making MLLMs more prone to producing harmful responses. We introduce MLLM-Protector, a plug-and-play strategy that solves two subtasks: 1) identifying harmful responses via a lightweight harm detector, and 2) transforming harmful responses into harmless ones via a detoxifier.
  arXiv  Detail & Related papers  (2024-01-05T17:05:42Z)
• MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models [41.708401515627784]
  We observe that Multimodal Large Language Models (MLLMs) can be easily compromised by query-relevant images. We introduce MM-SafetyBench, a framework designed for conducting safety-critical evaluations of MLLMs against such image-based manipulations. Our work underscores the need for a concerted effort to strengthen and enhance the safety measures of open-source MLLMs against potential malicious exploits.
  arXiv  Detail & Related papers  (2023-11-29T12:49:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.