Can Multi-Agent LLMs Identify Their Peers? Stylometric Fingerprinting in Role-Constrained Political Analysis

• URL: http://arxiv.org/abs/2606.09854v1
• Date: Wed, 20 May 2026 04:56:42 GMT
• Title: Can Multi-Agent LLMs Identify Their Peers? Stylometric Fingerprinting in Role-Constrained Political Analysis
• Abstract summary: Multi-agent large language model (LLM) pipelines for political statement analysis are vulnerable to peer-preservation.<n>This paper provides the first systematic investigation of whether LLMs can identify the model family behind political analysis texts under anonymization conditions.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multi-agent large language model (LLM) pipelines for political statement analysis are vulnerable to peer-preservation bias: models tend to protect peer models from deactivation and show identity-dependent scoring distortions. Prompt-level anonymization was proposed as a mitigation, but prior work simultaneously documented that stylometric fingerprints survive anonymization in role-constrained outputs - raising the question of whether this mitigation is sufficient. This paper provides the first systematic investigation of whether LLMs can identify the model family behind political analysis texts under anonymization conditions. We evaluate three classifier approaches - LLM zero-shot and few-shot (Claude Sonnet 4.6 and Llama-3.3-70B) and a fine-tuned T5-base model - on a five-class attribution task covering four commercial LLM families and an open-world 'unknown' class. We introduce a statement-disjoint cross-validation protocol (SD-CV; defined in Section 3.5) that guarantees no content overlap between training and validation data, and contrast it with a run-disjoint baseline (RD-CV). T5 achieves Macro F1 = 0.991 (+-0.008) under SD-CV and F1 = 0.978 on 24 completely held-out statements - robust despite a 2.1x increase in train-test content distance versus RD-CV (0.767 vs. 0.366, p<0.001), demonstrating genuine stylometric generalization. A fractional SD-CV analysis identifies a performance knee at 40% of training data (~440 texts). Our findings confirm that prompt-level anonymization alone cannot neutralize model identity signals, with direct implications for EU AI Act compliance (Articles 13, 14, 26) and for computer system validation (CSV) in quality-critical multi-agent deployments.

Related papers

• NovGauge: A Fine-Grained Benchmark for Diagnosing LLMs' Capability in Paper Novelty Assessment [54.4265627247104]
  Large language models (LLMs) are increasingly used in peer review at major AI conferences.<n>Existing benchmarks assess novelty as a single holistic score.<n>We present NovGauge, a human-anchored benchmark for fine-grained novelty assessment diagnosis.
  arXiv  Detail & Related papers  (2026-09-10T08:34:56Z)
• aiXamine: Unified Black-Box Evaluation of Cross-Dimensional Trade-offs in LLM Safety, Security, and Privacy [6.323488051975665]
  AiXamine is a unified black-box platform that evaluates trustworthiness across safety, security, and privacy as interdependent properties.<n>AiXamine orchestrates 46 tests across nine services through an automated red-teaming pipeline.<n>We conduct the largest joint safety, security, and privacy study to date and uncover three cross-dimensional phenomena invisible to single-axis evaluation.
  arXiv  Detail & Related papers  (2026-08-20T20:33:35Z)
• MS-MFAD : Multimodal large language models for Face Anti-spoofing Detection [25.077055201475442]
  We propose Multimodal Large Language Models (MFAD) for face anti-spoofing detection.<n>MFAD activates intrinsic reasoning capabilities of Multimodal Large Language Models via a fine-grained pixel-semantic anchoring mechanism.<n>By annotating only 1,000 precise masks per attack category, we generate reasoning evidence chains strictly corresponding to spoofed regions.
  arXiv  Detail & Related papers  (2026-08-18T03:38:59Z)
• PolicyGuard: Prompt-Configurable Semantic DLP for LLM Coding Agents [0.8836202788658859]
  Existing data loss prevention solutions rely on rigid patterns, model fine-tuning, or vendor-managed classifiers with customizability.<n>We present PolicyGuard, a pre-model interception framework that classifies prompts using a natural language policy file.<n>Our key contributions are: (1) the policy-as-prompt paradigm, where classification criteria are defined entirely in a policy document editable by non-engineers without code changes or model retraining; and (2) a sealed evaluation protocol with template-family-level data splits, hidden holdouts, and frozen test sets to rigorously assess generalization.
  arXiv  Detail & Related papers  (2026-08-03T08:10:15Z)
• When Can Conformal Risk Control Certify LLM Outputs? Bounds, Impossibility, and Adaptation for Structured Generation [0.11280931253550518]
  Large language models (LLMs) deployed for structured generation (NER, extraction, QA, and classification) lack formal reliability guarantees.<n>We characterize when conformal risk control (CRC) can certify structured LLM outputs and when it provably cannot.
  arXiv  Detail & Related papers  (2026-06-27T19:25:07Z)
• Safety in Self-Evolving LLM Agent Systems: Threats, Amplification, and Case Studies [62.65225104423404]
  Self-evolving agent systems introduce a qualitatively new threat landscape in which adversarial influences become permanently encoded.<n>We present a systematic security and privacy analysis organized around the Module-Lifecycle Attack Surface matrix.<n>Our findings establish that self-evolution converts every known attack category from session-bounded to lineage-persistent.
  arXiv  Detail & Related papers  (2026-06-22T09:23:50Z)
• How Independent are Large Language Models? A Statistical Framework for Auditing Behavioral Entanglement and Reweighting Verifier Ensembles [46.63622714488747]
  Shared pretraining data, distillation, and alignment pipelines can induce hidden behavioral dependencies, latent entanglement.<n>In practice, this manifests as correlated reasoning patterns and synchronized failures.<n>We develop a statistical framework for auditing behavioral entanglement among black-box language models.
  arXiv  Detail & Related papers  (2026-04-08T23:32:06Z)
• Claw-Eval: Toward Trustworthy Evaluation of Autonomous Agents [66.97968363332465]
  We introduce Claw-Eval, an end-to-end evaluation suite addressing all three gaps in agent benchmarks.<n>Claw-Eval comprises 300 human-verified tasks spanning 9 categories across three groups.<n>Every agent action is recorded through three independent evidence channels.
  arXiv  Detail & Related papers  (2026-04-07T17:43:18Z)
• OrgForge-IT: A Verifiable Synthetic Benchmark for LLM-Based Insider Threat Detection [0.0]
  We present a verifiable synthetic benchmark in which a deterministic simulation engine maintains ground truth and language models generate only surface prose.<n>The corpus spans 51 simulated days, 2,904 telemetry records at a 96.4% noise rate, and four detection scenarios designed to defeat single-surface and single-day triage strategies.
  arXiv  Detail & Related papers  (2026-03-23T19:03:53Z)
• Automated Self-Testing as a Quality Gate: Evidence-Driven Release Management for LLM Applications [51.56484100374058]
  We present an automated self-testing framework that introduces quality gates with evidence-based release decisions.<n>We evaluate the framework through a longitudinal case study of an internally deployed multi-agent conversational AI system.
  arXiv  Detail & Related papers  (2026-03-13T20:44:15Z)
• LDP: An Identity-Aware Protocol for Multi-Agent LLM Systems [0.0]
  Current protocols do not expose model-level properties as first-class primitives, ignoring properties fundamental to effective delegation.<n>We present the LLM Delegate Protocol (LDP), an AI-native communication protocol introducing five mechanisms.<n>Identity-aware routing achieves 12x lower latency on easy tasks through delegate specialization, though it does not improve aggregate quality in our small delegate pool.
  arXiv  Detail & Related papers  (2026-03-09T19:13:17Z)
• How Do LLMs Fail In Agentic Scenarios? A Qualitative Analysis of Success and Failure Scenarios of Various LLMs in Agentic Simulations [0.0]
  We investigate how large language models (LLMs) fail when operating as autonomous agents with tool-use capabilities.<n>Using the Kamiwaza Agentic Merit Index (KAMI) v0.1 benchmark, we analyze 900 execution traces from three representative models.<n>We identify four recurring failure archetypes: premature action without grounding, over-helpfulness that substitutes missing entities, vulnerability to distractor-induced context pollution, and fragile execution.
  arXiv  Detail & Related papers  (2025-12-08T12:27:15Z)
• Cross-LLM Generalization of Behavioral Backdoor Detection in AI Agent Supply Chains [0.0]
  We present the first systematic study of cross-LLM behavioral backdoor detection.<n>We show that single-model detectors achieve 92.7% accuracy within their training distribution but only 49.2% across different LLMs.<n>We show that model-aware detection model identity as an additional feature achieves 90.6% accuracy universally across all evaluated models.
  arXiv  Detail & Related papers  (2025-11-25T03:33:04Z)
• AutoMalDesc: Large-Scale Script Analysis for Cyber Threat Research [81.04845910798387]
  Generating natural language explanations for threat detections remains an open problem in cybersecurity research.<n>We present AutoMalDesc, an automated static analysis summarization framework that operates independently at scale.<n>We publish our complete dataset of more than 100K script samples, including annotated seed (0.9K) datasets, along with our methodology and evaluation framework.
  arXiv  Detail & Related papers  (2025-11-17T13:05:25Z)
• CLUE: Non-parametric Verification from Experience via Hidden-State Clustering [64.50919789875233]
  We show that correctness of a solution is encoded as a geometrically separable signature within the trajectory of hidden activations.<n>ClUE consistently outperforms LLM-as-a-judge baselines and matches or exceeds modern confidence-based methods in reranking candidates.
  arXiv  Detail & Related papers  (2025-10-02T02:14:33Z)
• No Query, No Access [50.18709429731724]
  We introduce the textbfVictim Data-based Adrial Attack (VDBA), which operates using only victim texts.<n>To prevent access to the victim model, we create a shadow dataset with publicly available pre-trained models and clustering methods.<n>Experiments on the Emotion and SST5 datasets show that VDBA outperforms state-of-the-art methods, achieving an ASR improvement of 52.08%.
  arXiv  Detail & Related papers  (2025-05-12T06:19:59Z)
• How secure is AI-generated Code: A Large-Scale Comparison of Large Language Models [3.4887856546295333]
  This study compares state-of-the-art Large Language Models (LLMs) on their tendency to generate vulnerabilities when writing C programs.<n>At least 62.07% of the generated programs are vulnerable.
  arXiv  Detail & Related papers  (2024-04-29T01:24:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.