Reasoning-Oriented Programming: Chaining Semantic Gadgets to Jailbreak Large Vision Language Models

• URL: http://arxiv.org/abs/2603.09246v1
• Date: Tue, 10 Mar 2026 06:18:48 GMT
• Title: Reasoning-Oriented Programming: Chaining Semantic Gadgets to Jailbreak Large Vision Language Models
• Abstract summary: Large Vision-Language Models (LVLMs) undergo safety alignment to suppress harmful content.<n>In this paper, we identify a systemic flaw where LVLMs can be induced to synthesize harmful logic from benign premises.<n>Our approach exploits the model's instruction-following capability to orchestrate a semantic collision of benign inputs.
• Score: 9.123902853709206
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Vision-Language Models (LVLMs) undergo safety alignment to suppress harmful content. However, current defenses predominantly target explicit malicious patterns in the input representation, often overlooking the vulnerabilities inherent in compositional reasoning. In this paper, we identify a systemic flaw where LVLMs can be induced to synthesize harmful logic from benign premises. We formalize this attack paradigm as \textit{Reasoning-Oriented Programming}, drawing a structural analogy to Return-Oriented Programming in systems security. Just as ROP circumvents memory protections by chaining benign instruction sequences, our approach exploits the model's instruction-following capability to orchestrate a semantic collision of orthogonal benign inputs. We instantiate this paradigm via \tool{}, an automated framework that optimizes for \textit{semantic orthogonality} and \textit{spatial isolation}. By generating visual gadgets that are semantically decoupled from the harmful intent and arranging them to prevent premature feature fusion, \tool{} forces the malicious logic to emerge only during the late-stage reasoning process. This effectively bypasses perception-level alignment. We evaluate \tool{} on SafeBench and MM-SafetyBench across 7 state-of-the-art 0.LVLMs, including GPT-4o and Claude 3.7 Sonnet. Our results demonstrate that \tool{} consistently circumvents safety alignment, outperforming the strongest existing baseline by an average of 4.67\% on open-source models and 9.50\% on commercial models.

Related papers

• Compiling Activation Steering into Weights via Null-Space Constraints for Stealthy Backdoors [48.881343993730844]
  Safety-aligned large language models (LLMs) are increasingly deployed in real-world pipelines.<n> adversaries can distribute backdoored checkpoints that behave normally under standard evaluation but jailbreak when a hidden trigger is present.<n>Recent post-hoc weight-editing methods offer an efficient approach to injecting such backdoors.
  arXiv  Detail & Related papers  (2026-04-14T06:48:33Z)
• Invisible Safety Threat: Malicious Finetuning for LLM via Steganography [74.00809267925642]
  A compromised large language model can maintain a facade of proper safety alignment while covertly generating harmful content.<n>We demonstrate this invisible safety threat on GPT-4.1 despite the OpenAI finetuning API's safeguards.<n>We quantitatively evaluate our method on the AdvBench dataset, using Llama-Guard-3-8B for content safety classification.
  arXiv  Detail & Related papers  (2026-03-09T08:48:27Z)
• Models as Lego Builders: Assembling Malice from Benign Blocks via Semantic Blueprints [9.698319722490043]
  We propose StructAttack, a simple yet effective single-query jailbreak framework under black-box settings.<n>We show that StructAttack exploits LVLMs' reasoning to assemble these slots into coherent harmful semantics.<n>Experiments on multiple models and benchmarks show the efficacy of our proposed StructAttack.
  arXiv  Detail & Related papers  (2026-03-08T11:22:41Z)
• The Compliance Paradox: Semantic-Instruction Decoupling in Automated Academic Code Evaluation [11.984098021215878]
  We introduce the Semantic-Preserving Adrial Code Injection (SPACI) Framework and the Abstract Syntax Tree-Aware Semantic Injection Protocol (AST-ASIP)<n>These methods exploit the Syntax-Semantics Gap by embedding adversarial directives into syntactically inert regions (trivia nodes) of the Abstract Syntax Tree.<n>Through a large-scale evaluation of 9 SOTA models across 25,000 submissions in Python, C, C++, and Java, we reveal catastrophic failure rates (>95%) in high-capacity open-weights models like DeepSeek-V3.
  arXiv  Detail & Related papers  (2026-01-29T07:40:58Z)
• How Does Prefix Matter in Reasoning Model Tuning? [57.69882799751655]
  We fine-tune three R1 series models across three core model capabilities: reasoning (mathematics), coding, safety, and factuality.<n>Results show that prefix-conditioned SFT improves both safety and reasoning performance, yielding up to +6% higher Safe@1 accuracy.
  arXiv  Detail & Related papers  (2026-01-04T18:04:23Z)
• In-Context Representation Hijacking [15.706479613839967]
  Doublespeak is an in-context representation hijacking attack against large language models.<n>We show that the substitution leads to the internal representation of the benign token converging toward that of the harmful one.<n>We show that this semantic emerges layer by layer, with benign meanings in early layers converging into harmful semantics in later ones.
  arXiv  Detail & Related papers  (2025-12-03T13:19:34Z)
• DiffuGuard: How Intrinsic Safety is Lost and Found in Diffusion Large Language Models [50.21378052667732]
  We conduct an in-depth analysis of dLLM vulnerabilities to jailbreak attacks across two distinct dimensions: intra-step and inter-step dynamics.<n>We propose DiffuGuard, a training-free defense framework that addresses vulnerabilities through a dual-stage approach.
  arXiv  Detail & Related papers  (2025-09-29T05:17:10Z)
• Trust Me, I Know This Function: Hijacking LLM Static Analysis using Bias [3.178301843099705]
  Large Language Models (LLMs) are increasingly trusted to perform automated code review and static analysis at scale.<n>This paper identifies and exploit a critical vulnerability in LLM-based code analysis.<n>We develop a fully automated, black-box algorithm that discovers and injects Familiar Pattern Attack (FPA) into target code.
  arXiv  Detail & Related papers  (2025-08-24T13:42:48Z)
• PRISM: Programmatic Reasoning with Image Sequence Manipulation for LVLM Jailbreaking [3.718606661938873]
  We propose a novel and effective jailbreak framework inspired by Return-Oriented Programming (ROP) techniques from software security.<n>Our approach decomposes a harmful instruction into a sequence of individually benign visual gadgets.<n>Our findings reveal a critical and underexplored vulnerability that exploits the compositional reasoning abilities of LVLMs.
  arXiv  Detail & Related papers  (2025-07-29T07:13:56Z)
• The Devil behind the mask: An emergent safety vulnerability of Diffusion LLMs [39.85609149662187]
  We present DIJA, the first systematic study and jailbreak attack framework that exploits unique safety weaknesses of dLLMs.<n>Our proposed DIJA constructs adversarial interleaved mask-text prompts that exploit the text generation mechanisms of dLLMs.<n>Our findings underscore the urgent need for rethinking safety alignment in this emerging class of language models.
  arXiv  Detail & Related papers  (2025-07-15T08:44:46Z)
• Revisiting Backdoor Attacks on LLMs: A Stealthy and Practical Poisoning Framework via Harmless Inputs [54.90315421117162]
  We propose a novel poisoning method via completely harmless data.<n>Inspired by the causal reasoning in auto-regressive LLMs, we aim to establish robust associations between triggers and an affirmative response prefix.<n>We observe an interesting resistance phenomenon where the LLM initially appears to agree but subsequently refuses to answer.
  arXiv  Detail & Related papers  (2025-05-23T08:13:59Z)
• What Makes and Breaks Safety Fine-tuning? A Mechanistic Study [64.9691741899956]
  Safety fine-tuning helps align Large Language Models (LLMs) with human preferences for their safe deployment. We design a synthetic data generation framework that captures salient aspects of an unsafe input. Using this, we investigate three well-known safety fine-tuning methods.
  arXiv  Detail & Related papers  (2024-07-14T16:12:57Z)
• AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Prompting [54.931241667414184]
  We propose textbfAdaptive textbfShield Prompting, which prepends inputs with defense prompts to defend MLLMs against structure-based jailbreak attacks. Our methods can consistently improve MLLMs' robustness against structure-based jailbreak attacks.
  arXiv  Detail & Related papers  (2024-03-14T15:57:13Z)
• Defending Large Language Models against Jailbreak Attacks via Semantic Smoothing [107.97160023681184]
  Aligned large language models (LLMs) are vulnerable to jailbreaking attacks. We propose SEMANTICSMOOTH, a smoothing-based defense that aggregates predictions of semantically transformed copies of a given input prompt.
  arXiv  Detail & Related papers  (2024-02-25T20:36:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.