Black-Box Forensics for Conversational LLM Agents

• URL: http://arxiv.org/abs/2606.22698v1
• Date: Sun, 21 Jun 2026 22:24:55 GMT
• Title: Black-Box Forensics for Conversational LLM Agents
• Abstract summary: Black-box forensics for conversational LLM agents offers a path to accountability for systems hidden behind anonymous endpoints.<n>We identify the base model behind an agent with 98% accuracy from a few turns of non-adversarial conversation.<n>Our cross-encoder fingerprinting method achieves an AUC of 0.768 and an F1 of 0.703 on entirely unseen system prompts.
• Score: 27.24002262427959
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As LLM-powered scams proliferate, black-box forensics for conversational LLM agents offers a path to accountability for systems hidden behind anonymous endpoints. Identifying the base model behind a chatbot endpoint (attribution), without model parameter access or knowledge of the hidden system prompt, would let investigators trace AI-enabled scams back to the providers whose models power them. Detecting when two endpoints run the exact same system prompt (fingerprinting), even one novel and unseen, would link individual scams into criminal networks and expose silent API changes. We conduct an empirical investigation of both capabilities. Our attribution classifiers identify the base model behind an agent with 98% accuracy from a few turns of non-adversarial conversation. Attribution of system prompts, while possible, requires retraining on a large amount of data for each prompt; system prompts in the wild are unbounded and ever-changing, making this approach costly. To tackle this more open-ended setting, our cross-encoder fingerprinting method achieves an AUC of 0.768 and an F1 of 0.703 on entirely unseen system prompts, and aggregating 50 interaction conversations from each target agent boosts AUC to 0.943. Conversational agents with unseen system prompts can thus be fingerprinted with robust accuracy from a few turns of ordinary conversation.

Related papers

• AgentProv: Auditing Agentic LLM API Providers via Tool-use Policy Probes [38.37599802008238]
  Commercial LLM APIs advertise a specific foundation model, but the served backbone may be silently substituted, quantized, or wrapped.<n>All existing audits decide backbone identity from the text-output channel.<n>We introduce Agentic Provenance (AgentProv), the first action-based identity audit for agentic LLM APIs.
  arXiv  Detail & Related papers  (2026-08-30T08:22:12Z)
• Do System Prompts Leave Behavioral Fingerprints? A Large-Scale Empirical Study of Clone Detection via Output Similarity [12.32748486531306]
  We propose Black-Box Behavioral Fingerprinting (BBF) for system prompts.<n>BBF registers a behavioral signature from model outputs and later tests whether a suspect deployment matches that signature.<n>BBF resists non-adaptive prompt paraphrasing (AUC $geq 0.889$) and is robust to imperfect extraction.
  arXiv  Detail & Related papers  (2026-08-25T12:12:22Z)
• When Collaboration Becomes a Trigger: Collective Evidence-Threshold Backdoors in Multi-Agent Systems [57.450132281381066]
  Backdoor behavior can be activated when peer evidence reaches a hidden threshold, rather than being determined by any single message.<n>We introduce a collective evidence-threshold backdoor paradigm for MAS and Boundary-Conditioned Backdoor Injection (BCBI)<n>We propose LAtent Transition Test-time Evaluation (LATTE), a clean-only latent-transition defense that learns benign communication dynamics and quarantines anomalous agent updates.
  arXiv  Detail & Related papers  (2026-08-02T08:38:44Z)
• An Explainable Agentic System for Detection of Conversational Scams with Summary-Based Memory [0.7712979281505654]
  This paper extends single-message phishing detection and presents an explainable agentic system for detecting sophisticated conversational scams.<n>ConScamBench-278 is an initial public multi-category benchmark for conversational scam detection spanning eight scam types.<n>On isolated messages the single-message detector attains 100% phishing recall, while the conversation-level detector identifies all conversational scams in the public LoveFraud02 corpus.
  arXiv  Detail & Related papers  (2026-07-13T15:36:19Z)
• Decoding Hidden Deception in Reasoning LLMs: Activation Explainers for Deception Auditing [22.30404793367838]
  We introduce STATEWITNESS, an activation explainer for deception auditing.<n>A separate decoder reads a target model's hidden states, then answers natural-language queries or emits structured reports about them.<n>We evaluate STATEWITNESS on two target reasoning LLMs across seven deception datasets.
  arXiv  Detail & Related papers  (2026-06-16T03:41:29Z)
• Human-Centered LLM-Agent System for Detecting Anomalous Digital Asset Transactions [0.0]
  We present HCLA, a human-centered multi-agent system for anomaly detection in digital asset transactions.<n>The system links three roles: Parsing, Detection, and Explanation, into a conversational workflow that lets non-experts ask questions in natural language.<n>We describe the architecture, interaction loop, dataset, evaluation protocol, and limitations, and discuss how a human-in-the-loop design improves transparency and trust in financial forensics.
  arXiv  Detail & Related papers  (2025-10-23T01:04:36Z)
• Mind the Goal: Data-Efficient Goal-Oriented Evaluation of Conversational Agents and Chatbots using Teacher Models [0.0]
  We propose a comprehensive framework for goal-oriented evaluation of multi-agent systems.<n>Our method segments conversations by user goals and evaluates success using all relevant turns.<n>In an enterprise setting, we apply our framework to evaluate AIDA, a zero-to-one employee conversational agent system.
  arXiv  Detail & Related papers  (2025-10-04T06:22:47Z)
• MCP-Orchestrated Multi-Agent System for Automated Disinformation Detection [84.75972919995398]
  This paper presents a multi-agent system that uses relation extraction to detect disinformation in news articles.<n>The proposed Agentic AI system combines four agents: (i) a machine learning agent (logistic regression), (ii) a Wikipedia knowledge check agent, and (iv) a web-scraped data analyzer.<n>Results demonstrate that the multi-agent ensemble achieves 95.3% accuracy with an F1 score of 0.964, significantly outperforming individual agents and traditional approaches.
  arXiv  Detail & Related papers  (2025-08-13T19:14:48Z)
• Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems [50.29939179830491]
  Failure attribution in LLM multi-agent systems remains underexplored and labor-intensive.<n>We develop and evaluate three automated failure attribution methods, summarizing their corresponding pros and cons.<n>The best method achieves 53.5% accuracy in identifying failure-responsible agents but only 14.2% in pinpointing failure steps.
  arXiv  Detail & Related papers  (2025-04-30T23:09:44Z)
• LeakAgent: RL-based Red-teaming Agent for LLM Privacy Leakage [78.33839735526769]
  LeakAgent is a novel black-box red-teaming framework for privacy leakage.<n>Our framework trains an open-source LLM through reinforcement learning as the attack agent to generate adversarial prompts.<n>We show that LeakAgent significantly outperforms existing rule-based approaches in training data extraction and automated methods in system prompt leakage.
  arXiv  Detail & Related papers  (2024-12-07T20:09:01Z)
• Imprompter: Tricking LLM Agents into Improper Tool Use [35.255462653237885]
  Large Language Model (LLM) Agents are an emerging computing paradigm that blends generative machine learning with tools such as code interpreters, web browsing, email, and more generally, external resources. We contribute to the security foundations of agent-based systems and surface a new class of automatically computed obfuscated adversarial prompt attacks.
  arXiv  Detail & Related papers  (2024-10-19T01:00:57Z)
• Token-Level Adversarial Prompt Detection Based on Perplexity Measures and Contextual Information [67.78183175605761]
  Large Language Models are susceptible to adversarial prompt attacks. This vulnerability underscores a significant concern regarding the robustness and reliability of LLMs. We introduce a novel approach to detecting adversarial prompts at a token level.
  arXiv  Detail & Related papers  (2023-11-20T03:17:21Z)
• Language Models as Black-Box Optimizers for Vision-Language Models [62.80817942316398]
  Vision-language models (VLMs) pre-trained on web-scale datasets have demonstrated remarkable capabilities on downstream tasks when fine-tuned with minimal data. We aim to develop a black-box approach to optimize VLMs through natural language prompts.
  arXiv  Detail & Related papers  (2023-09-12T04:03:41Z)
• Guiding Large Language Models via Directional Stimulus Prompting [114.84930073977672]
  We introduce Directional Stimulus Prompting, a novel framework for guiding black-box large language models (LLMs) toward specific desired outputs. Instead of directly adjusting LLMs, our method employs a small tunable policy model to generate an auxiliary directional stimulus prompt for each input instance.
  arXiv  Detail & Related papers  (2023-02-22T17:44:15Z)
• Self-Prompting Large Language Models for Zero-Shot Open-Domain QA [67.08732962244301]
  Open-Domain Question Answering (ODQA) aims to answer questions without explicitly providing background documents. This task becomes notably challenging in a zero-shot setting where no data is available to train tailored retrieval-reader models. We propose a Self-Prompting framework to explicitly utilize the massive knowledge encoded in the parameters of Large Language Models.
  arXiv  Detail & Related papers  (2022-12-16T18:23:43Z)
• What is wrong with you?: Leveraging User Sentiment for Automatic Dialog Evaluation [73.03318027164605]
  We propose to use information that can be automatically extracted from the next user utterance as a proxy to measure the quality of the previous system response. Our model generalizes across both spoken and written open-domain dialog corpora collected from real and paid users.
  arXiv  Detail & Related papers  (2022-03-25T22:09:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.