Detecting High-Stakes Interactions with Activation Probes

• URL: http://arxiv.org/abs/2506.10805v2
• Date: Fri, 13 Jun 2025 09:04:48 GMT
• Title: Detecting High-Stakes Interactions with Activation Probes
• Abstract summary: This paper examines activation probes for detecting "high-stakes" interactions, where the text indicates that the interaction might lead to significant harm.<n>We evaluate several probe architectures trained on synthetic data, and find them to exhibit robust generalization to diverse, out-of-distribution, real-world data.<n>Our experiments also highlight the potential of building resource-aware hierarchical monitoring systems.
• Score: 10.421494494629421
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Monitoring is an important aspect of safely deploying Large Language Models (LLMs). This paper examines activation probes for detecting "high-stakes" interactions -- where the text indicates that the interaction might lead to significant harm -- as a critical, yet underexplored, target for such monitoring. We evaluate several probe architectures trained on synthetic data, and find them to exhibit robust generalization to diverse, out-of-distribution, real-world data. Probes' performance is comparable to that of prompted or finetuned medium-sized LLM monitors, while offering computational savings of six orders-of-magnitude. Our experiments also highlight the potential of building resource-aware hierarchical monitoring systems, where probes serve as an efficient initial filter and flag cases for more expensive downstream analysis. We release our novel synthetic dataset and codebase to encourage further study.

Related papers

• $D^2$-Monitor: Dynamic Safety Monitoring for Diffusion LLMs via Hesitation-Aware Routing [63.49501120848927]
  Safety monitoring for large language models (D-LLMs) remains largely unexplored.<n>We propose $D2$-Monitor, a bi-level safety monitor for D-LLMs.<n>$D2$-Monitor adopts a lightweight probe as an always-on monitor to jointly estimate hesitation and perform base classification.
  arXiv  Detail & Related papers  (2026-05-25T14:22:21Z)
• NeuroTrace: Inference Provenance-Based Detection of Adversarial Examples [1.096626056612224]
  We introduce NeuroTrace, a framework for analyzing inference provenance through Inference Provenance Graphs (IPGs)<n>IPGs are heterogeneous graphs that capture both activation behavior and parameter-induced dataflow during a model's forward pass.<n>We evaluate IPG-based detectors for adversarial example detection under intra-attack, multi-attack, and cross-threat transfer settings.
  arXiv  Detail & Related papers  (2026-04-15T22:23:40Z)
• Heuristic-inspired Reasoning Priors Facilitate Data-Efficient Referring Object Detection [53.988759250627425]
  HeROD (Heuristic-inspired ROD) is a lightweight, model-agnostic framework that injects explicit, interpretable spatial and semantic reasonings.<n>HeROD consistently outperforms strong grounding baselines in scarcelabel regimes.
  arXiv  Detail & Related papers  (2026-03-25T10:33:22Z)
• Explainable Deepfake Detection with RL Enhanced Self-Blended Images [13.456365650507877]
  We propose an automated Chain-of-Thought (CoT) data generation framework based on Self-Blended Images, along with an RL-enhanced deepfake detection framework.<n>Our method achieves performance competitive with state-of-the-art (SOTA) approaches across multiple cross-dataset benchmarks.
  arXiv  Detail & Related papers  (2026-01-22T03:55:46Z)
• From Words to Wavelengths: VLMs for Few-Shot Multispectral Object Detection [7.459632891054827]
  Multispectral object detection is critical for safety-sensitive applications such as autonomous driving and surveillance.<n>Motivated by the recent success of Vision-Language Models in computer vision, we explore their potential for few-shot multispectral object detection.
  arXiv  Detail & Related papers  (2025-12-17T21:06:36Z)
• VILOD: A Visual Interactive Labeling Tool for Object Detection [0.0]
  This thesis develops and investigates "VILOD: A Visual Interactive Labeling tool for Object Detection"<n>It enables users to explore data, interpret model states, AL suggestions, and implement diverse sample selection strategies within an iterative HITL workflow for Object Detection.<n>The study showed that different visually-guided labeling strategies employed within VILOD result in competitive OD performance trajectories.
  arXiv  Detail & Related papers  (2025-08-29T19:27:10Z)
• Probing Deep into Temporal Profile Makes the Infrared Small Target Detector Much Better [63.567886330598945]
  Infrared small target (IRST) detection is challenging in simultaneously achieving precise, universal, robust and efficient performance.<n>Current learning-based methods attempt to leverage more" information from both the spatial and the short-term temporal domains.<n>We propose an efficient deep temporal probe network (DeepPro) that only performs calculations in the time dimension for IRST detection.
  arXiv  Detail & Related papers  (2025-06-15T08:19:32Z)
• Hallucination Detection in LLMs with Topological Divergence on Attention Graphs [64.74977204942199]
  Hallucination, i.e., generating factually incorrect content, remains a critical challenge for large language models.<n>We introduce TOHA, a TOpology-based HAllucination detector in the RAG setting.
  arXiv  Detail & Related papers  (2025-04-14T10:06:27Z)
• Oriented Tiny Object Detection: A Dataset, Benchmark, and Dynamic Unbiased Learning [51.170479006249195]
  We introduce a new dataset, benchmark, and a dynamic coarse-to-fine learning scheme in this study.<n>Our proposed dataset, AI-TOD-R, features the smallest object sizes among all oriented object detection datasets.<n>We present a benchmark spanning a broad range of detection paradigms, including both fully-supervised and label-efficient approaches.
  arXiv  Detail & Related papers  (2024-12-16T09:14:32Z)
• How Important are Data Augmentations to Close the Domain Gap for Object Detection in Orbit? [15.550663626482903]
  We investigate the efficacy of data augmentations to close the domain gap in spaceborne computer vision. We propose two novel data augmentations specifically developed to emulate the visual effects observed in orbital imagery.
  arXiv  Detail & Related papers  (2024-10-21T08:24:46Z)
• Efficient Parameter Mining and Freezing for Continual Object Detection [0.0]
  We propose efficient ways to identify which layers are the most important for a network to maintain the performance of a detector across sequential updates. The presented findings highlight the substantial advantages of layer-level parameter isolation in facilitating incremental learning within object detection models.
  arXiv  Detail & Related papers  (2024-02-20T01:07:32Z)
• Active Sensing with Predictive Coding and Uncertainty Minimization [0.0]
  We present an end-to-end procedure for embodied exploration inspired by two biological computations. We first demonstrate our approach in a maze navigation task and show that it can discover the underlying transition distributions and spatial features of the environment. We show that our model builds unsupervised representations through exploration that allow it to efficiently categorize visual scenes.
  arXiv  Detail & Related papers  (2023-07-02T21:14:49Z)
• Exploiting Multimodal Synthetic Data for Egocentric Human-Object Interaction Detection in an Industrial Scenario [14.188006024550257]
  EgoISM-HOI is a new multimodal dataset composed of synthetic EHOI images in an industrial environment with rich annotations of hands and objects. Our study shows that exploiting synthetic data to pre-train the proposed method significantly improves performance when tested on real-world data. To support research in this field, we publicly release the datasets, source code, and pre-trained models at https://iplab.dmi.unict.it/egoism-hoi.
  arXiv  Detail & Related papers  (2023-06-21T09:56:55Z)
• AntPivot: Livestream Highlight Detection via Hierarchical Attention Mechanism [64.70568612993416]
  We formulate a new task Livestream Highlight Detection, discuss and analyze the difficulties listed above and propose a novel architecture AntPivot to solve this problem. We construct a fully-annotated dataset AntHighlight to instantiate this task and evaluate the performance of our model.
  arXiv  Detail & Related papers  (2022-06-10T05:58:11Z)
• Egocentric Human-Object Interaction Detection Exploiting Synthetic Data [19.220651860718892]
  We consider the problem of detecting Egocentric HumanObject Interactions (EHOIs) in industrial contexts. We propose a pipeline and a tool to generate photo-realistic synthetic First Person Vision (FPV) images automatically labeled for EHOI detection.
  arXiv  Detail & Related papers  (2022-04-14T15:59:15Z)
• WSSOD: A New Pipeline for Weakly- and Semi-Supervised Object Detection [75.80075054706079]
  We propose a weakly- and semi-supervised object detection framework (WSSOD) An agent detector is first trained on a joint dataset and then used to predict pseudo bounding boxes on weakly-annotated images. The proposed framework demonstrates remarkable performance on PASCAL-VOC and MSCOCO benchmark, achieving a high performance comparable to those obtained in fully-supervised settings.
  arXiv  Detail & Related papers  (2021-05-21T11:58:50Z)
• Provably Efficient Causal Reinforcement Learning with Confounded Observational Data [135.64775986546505]
  We study how to incorporate the dataset (observational data) collected offline, which is often abundantly available in practice, to improve the sample efficiency in the online setting. We propose the deconfounded optimistic value iteration (DOVI) algorithm, which incorporates the confounded observational data in a provably efficient manner.
  arXiv  Detail & Related papers  (2020-06-22T14:49:33Z)
• Stance Detection Benchmark: How Robust Is Your Stance Detection? [65.91772010586605]
  Stance Detection (StD) aims to detect an author's stance towards a certain topic or claim. We introduce a StD benchmark that learns from ten StD datasets of various domains in a multi-dataset learning setting. Within this benchmark setup, we are able to present new state-of-the-art results on five of the datasets.
  arXiv  Detail & Related papers  (2020-01-06T13:37:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.