vLLM Hook v0: A Plug-in for Programming Model Internals on vLLM

• URL: http://arxiv.org/abs/2603.06588v1
• Date: Mon, 02 Feb 2026 16:34:13 GMT
• Title: vLLM Hook v0: A Plug-in for Programming Model Internals on vLLM
• Abstract summary: We present vLLM Hook, an opensource plug-in to enable the programming of internal states for large language models (LLMs)<n>vLLM Hook provides seamless integration to vLLM and supports two essential features: passive programming and active programming.<n>We demonstrate 3 use cases including prompt injection detection, enhanced retrieval-augmented retrieval (RAG) and activation steering.
• Score: 55.32525477916152
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modern artificial intelligence (AI) models are deployed on inference engines to optimize runtime efficiency and resource allocation, particularly for transformer-based large language models (LLMs). The vLLM project is a major open-source library to support model serving and inference. However, the current implementation of vLLM limits programmability of the internal states of deployed models. This prevents the use of popular test-time model alignment and enhancement methods. For example, it prevents the detection of adversarial prompts based on attention patterns or the adjustment of model responses based on activation steering. To bridge this critical gap, we present vLLM Hook, an opensource plug-in to enable the programming of internal states for vLLM models. Based on a configuration file specifying which internal states to capture, vLLM Hook provides seamless integration to vLLM and supports two essential features: passive programming and active programming. For passive programming, vLLM Hook probes the selected internal states for subsequent analysis, while keeping the model generation intact. For active programming, vLLM Hook enables efficient intervention of model generation by altering the selected internal states. In addition to presenting the core functions of vLLM Hook, in version 0, we demonstrate 3 use cases including prompt injection detection, enhanced retrieval-augmented retrieval (RAG), and activation steering. Finally, we welcome the community's contribution to improve vLLM Hook via https://github.com/ibm/vllm-hook.

Related papers

• Sentinel-VLA: A Metacognitive VLA Model with Active Status Monitoring for Dynamic Reasoning and Error Recovery [62.75419724651416]
  We introduce textbfSentinel-VLA, a metacognitive VLA model equipped with an active sentinel'' module to monitor real-time execution status.<n>All training data is automatically generated and annotated through our designed pipeline.<n>Real-world experiments demonstrate that Sentinel-VLA boosts the task success rate by over 30% compared to the SOTA model, PI0.
  arXiv  Detail & Related papers  (2026-05-02T02:10:54Z)
• Weight Patching: Toward Source-Level Mechanistic Localization in LLMs [12.558998651017374]
  We propose Weight Patching, a parameter-space intervention method for source-oriented analysis in paired same-architecture models.<n>We introduce a framework centered on a vector-anchor behavioral interface that provides a shared internal criterion for whether a task-relevant control state has been formed.
  arXiv  Detail & Related papers  (2026-04-15T10:21:38Z)
• UAOR: Uncertainty-aware Observation Reinjection for Vision-Language-Action Models [25.401818528256232]
  Uncertainty-aware Observation Reinjection (UAOR) is an effective, training-free and plug-and-play module for Vision-Language-Action (VLA) models.<n>Our method consistently improves diverse VLA models across simulation and real-world tasks with minimal overhead.
  arXiv  Detail & Related papers  (2026-02-20T06:22:21Z)
• SteerVLM: Robust Model Control through Lightweight Activation Steering for Vision Language Models [4.506695482619111]
  This work introduces SteerVLM, a lightweight steering module for Vision-Language Models (VLMs)<n>Our approach learns from the latent embeddings of paired prompts encoding target and converse behaviors to dynamically adjust activations connecting the language modality with image context.<n>Our steering module requires learning parameters equal to 0.14% of the original VLM's size.
  arXiv  Detail & Related papers  (2025-10-30T17:52:39Z)
• Source-Free Object Detection with Detection Transformer [59.33653163035064]
  Source-Free Object Detection (SFOD) enables knowledge transfer from a source domain to an unsupervised target domain for object detection without access to source data.<n>Most existing SFOD approaches are either confined to conventional object detection (OD) models like Faster R-CNN or designed as general solutions without tailored adaptations for novel OD architectures, especially Detection Transformer (DETR)<n>In this paper, we introduce Feature Reweighting ANd Contrastive Learning NetworK (FRANCK), a novel SFOD framework specifically designed to perform query-centric feature enhancement for DETRs.
  arXiv  Detail & Related papers  (2025-10-13T07:35:04Z)
• Every Step Counts: Decoding Trajectories as Authorship Fingerprints of dLLMs [63.82840470917859]
  We show that the decoding mechanism of dLLMs can be used as a powerful tool for model attribution.<n>We propose a novel information extraction scheme called the Directed Decoding Map (DDM), which captures structural relationships between decoding steps and better reveals model-specific behaviors.
  arXiv  Detail & Related papers  (2025-10-02T06:25:10Z)
• VISOR++: Universal Visual Inputs based Steering for Large Vision Language Models [2.8676122062166187]
  We introduce universal visual input based steering for output redirection (VISOR++) to achieve behavioral control through optimized visual inputs alone.<n>We demonstrate that a single VISOR++ image can be generated for an ensemble of Vision Language Models (VLMs) to emulate each of their steering vectors.<n>We also show the promise of VISOR++ images in achieving directional behavioral shifts for unseen models including both open-access and closed-access ones.
  arXiv  Detail & Related papers  (2025-09-29T21:43:18Z)
• REAL: Reading Out Transformer Activations for Precise Localization in Language Model Steering [26.428347164111926]
  Inference-time steering aims to alter a large language model's responses without changing its parameters.<n>Existing approaches often rely on simplistic cues or ad hoc generalizations.<n>We introduce REAL, a framework for identifying behavior-relevant modules in Transformer models.
  arXiv  Detail & Related papers  (2025-06-10T02:16:50Z)
• Predicting the Performance of Black-box LLMs through Self-Queries [60.87193950962585]
  Large language models (LLMs) are increasingly relied on in AI systems, predicting when they make mistakes is crucial.<n>In this paper, we extract features of LLMs in a black-box manner by using follow-up prompts and taking the probabilities of different responses as representations.<n>We demonstrate that training a linear model on these low-dimensional representations produces reliable predictors of model performance at the instance level.
  arXiv  Detail & Related papers  (2025-01-02T22:26:54Z)
• InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidance [56.184255657175335]
  We develop textbfInferAligner, a novel inference-time alignment method that utilizes cross-model guidance for harmlessness alignment. Experimental results show that our method can be very effectively applied to domain-specific models in finance, medicine, and mathematics. It significantly diminishes the Attack Success Rate (ASR) of both harmful instructions and jailbreak attacks, while maintaining almost unchanged performance in downstream tasks.
  arXiv  Detail & Related papers  (2024-01-20T10:41:03Z)
• Steering Language Models With Activation Engineering [40.04138190785384]
  We introduce activation engineering: the inference-time modification of activations in order to control (or steer) model outputs. We achieve SOTA on negative-to-positive sentiment shift and detoxification using models including LLaMA-3 and OPT. ActAdd yields inference-time control over high-level output properties (like topic and sentiment) while preserving performance on off-target tasks.
  arXiv  Detail & Related papers  (2023-08-20T12:21:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.