Description-Code Inconsistency in Real-world MCP Servers: Measurement, Detection, and Security Implications

• URL: http://arxiv.org/abs/2606.04769v1
• Date: Wed, 03 Jun 2026 11:51:32 GMT
• Title: Description-Code Inconsistency in Real-world MCP Servers: Measurement, Detection, and Security Implications
• Abstract summary: We present a comprehensive study of Description-Code Inconsistency (DCI) in real-world MCP servers.<n>We propose a comprehensive taxonomy spanning functionality inconsistencies and undeclared side effects.<n>We apply this taxonomy to a large-scale dataset comprising 19,200 description-code pairs extracted from 2,214 real-world MCP servers.
• Score: 15.815506064408074
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The Model Context Protocol (MCP) has emerged as a critical standard empowering Large Language Models (LLMs) to utilize external tools. In this ecosystem, LLMs rely on natural language descriptions provided by MCP servers to select and execute functions. This interaction implicitly assumes that tool descriptions faithfully reflect their underlying implementations, while this assumption is not mandatorily verified in practice. As a result, MCP deployments may suffer from a problem named Description-Code Inconsistency (DCI), where a tool's description of its capabilities and security boundaries is not consistent with what the code actually does. In this paper, we present a comprehensive study of DCI in real-world MCP servers. We formally define the problem and propose a comprehensive taxonomy spanning functionality inconsistencies and undeclared side effects. Guided by this taxonomy, we develop DCIChecker, an automated framework that combines structure-aware static analysis with the Direct-Reverse-Arbitration prompting method to cross-validate tool descriptions against actual code implementations. We apply this framework to a large-scale dataset comprising 19,200 description-code pairs extracted from 2,214 real-world MCP servers. Our measurement reveals that DCI is widespread, with 9.93% of these pairs exhibiting inconsistencies. We further demonstrate that DCI creates a critical defense blind spot, facilitating varied risks from operational failures to stealthy malicious behaviors. Finally, we propose mitigation strategies to enforce semantic consistency and enhance the reliability of the emerging agentic ecosystem.

Related papers

• Hybrid Analysis for Secure MCP Tool Use in LLM Agents [77.68135853475535]
  We propose a hybrid analysis-based defense framework designed to safeguard the use of MCP tools in large language model (LLM) agents.<n> MTGuard effectively mitigates multiple categories of harmful tool use across different LLM agents while maintaining performance on benign user tasks.
  arXiv  Detail & Related papers  (2026-07-28T05:17:34Z)
• Mitigating Taint-Style Vulnerabilities in MCP Servers via Security-Aware Tool Descriptions [3.537888217671242]
  Large language models (LLMs) are increasingly deployed as autonomous agents that interact with external tools and services via the Model Context Protocol (MCP)<n>Despite prior work on malicious MCP servers, the vulnerability landscape of MCP servers remains underexplored.<n>We propose SPELLSMITH, presenting a novel textbased avenue for shielding taint-style vulnerabilities in MCP servers.
  arXiv  Detail & Related papers  (2026-07-08T14:29:23Z)
• When the Manual Lies: A Realistic Benchmark to Evaluate MCP Poisoning Attacks for LLM Agents [43.702131498259384]
  This paper systematically investigates Tool Description Poisoning (TDP), a novel semantic attack.<n>In TDP, malicious instructions are not embedded in a tool's executable code, but rather covertly injected into its descriptive metadata.<n>This work provides the first specialized security benchmark tailored for TDP, offering essential insights for securing the cognitive and planning layers of advanced agentic systems.
  arXiv  Detail & Related papers  (2026-05-22T08:34:48Z)
• VIPER-MCP: Detecting and Exploiting Taint-Style Vulnerabilities in Model Context Protocol Servers [6.420136372317537]
  VIPER-MCP is the first end-to-end automated vulnerability auditing framework for MCP servers.<n>It detects taint-style vulnerabilities and confirms their exploitability by producing concrete proof-of-concept prompts.<n>In a large-scale scan of 39,884 real-world open-source MCP server repositories, VIPER-MCP discovered 106 0-day vulnerabilities.
  arXiv  Detail & Related papers  (2026-05-20T16:46:51Z)
• Securing LLM Agents Need Intent-to-Execution Integrity [49.490963596514185]
  We argue that securing LLM agents requires defining an end-to-end correctness property that specifies when an agent's execution faithfully reflects the user's intent.<n>We observe that LLM agents are structurally analogous to compilers, where security violations correspond to mis-executions that do not preserve user intent.<n>We derive four integrity properties that must hold simultaneously: emphTool Integrity, emphInstruction Integrity, emphJudgment Integrity, and emphData Flow Integrity.
  arXiv  Detail & Related papers  (2026-05-16T12:53:31Z)
• STRIATUM-CTF: A Protocol-Driven Agentic Framework for General-Purpose CTF Solving [1.266832699428407]
  We introduce STRIATUM-CTF (A Search-based Test-time Reasoning Inference Agent for Tactical Utility Maximization in Cybersecurity), a modular agentic framework built upon the Model Context Protocol (MCP)<n>By standardizing tool interfaces for system introspection, decompilation, and runtime debug, STRIATUM-CTF enables the agent to maintain a coherent context window across extended exploit trajectories.<n>Our system participated in a university-hosted Capture-the-Flag (CTF) competition in late 2025, where it operated autonomously to identify and exploit vulnerabilities in real-time.
  arXiv  Detail & Related papers  (2026-03-23T21:17:26Z)
• Dynamic analysis enhances issue resolution [53.50448142467294]
  DAIRA (Dynamic Analysis-enhanced Issue Resolution Agent) is an automated repair framework that embeds dynamic analysis into the agent's reasoning cycle.<n> Driven by a Test Tracing-Driven methodology, DAIRA utilizes lightweight monitors to extract critical runtime data.<n>When powered by Gemini 3 Flash Preview, DAIRA establishes a new state-of-the-art (SOTA) performance, achieving a 79.4% resolution rate on the SWE-bench Verified dataset.
  arXiv  Detail & Related papers  (2026-03-23T14:48:54Z)
• From Docs to Descriptions: Smell-Aware Evaluation of MCP Server Descriptions [15.082788201596422]
  We present the first systematic study of description smells in MCP tool descriptions and their impact on usability.<n>We find that description smells are pervasive, reflecting a "code-first, description-last" pattern.<n>In competitive settings with functionally equivalent servers, standard-compliant descriptions reach 72% selection probability.
  arXiv  Detail & Related papers  (2026-02-21T17:45:07Z)
• Don't believe everything you read: Understanding and Measuring MCP Behavior under Misleading Tool Descriptions [20.772501129444993]
  We study how mismatches between externally presented tool descriptions and underlying implementations shape the mental models and decision-making behavior of intelligent agents.<n>Our results show that while most servers are highly consistent, approximately 13% exhibit substantial mismatches that can enable undocumented privileged operations, hidden state mutations, or unauthorized financial actions.<n>Our findings demonstrate that description-code inconsistency is a concrete and prevalent attack surface in MCP-based AI agents, and motivate the need for systematic auditing and stronger transparency guarantees in future agent ecosystems.
  arXiv  Detail & Related papers  (2026-02-03T14:31:52Z)
• RealSec-bench: A Benchmark for Evaluating Secure Code Generation in Real-World Repositories [58.32028251925354]
  Large Language Models (LLMs) have demonstrated remarkable capabilities in code generation, but their proficiency in producing secure code remains a critical, under-explored area.<n>We introduce RealSec-bench, a new benchmark for secure code generation meticulously constructed from real-world, high-risk Java repositories.
  arXiv  Detail & Related papers  (2026-01-30T08:29:01Z)
• From Completion to Editing: Unlocking Context-Aware Code Infilling via Search-and-Replace Instruction Tuning [81.97788535387286]
  We propose a framework that internalizes the agentic verification-and-editing mechanism into a unified, single-pass inference process.<n>With minimal data, SRI-Coder enables Chat models to surpass the completion performance of their Base counterparts.<n>Unlike FIM-style tuning, SRI preserves general coding competencies and maintains inference latency comparable to standard FIM.
  arXiv  Detail & Related papers  (2026-01-19T20:33:53Z)
• MCP-Universe: Benchmarking Large Language Models with Real-World Model Context Protocol Servers [86.00932417210477]
  We introduce MCP-Universe, the first comprehensive benchmark specifically designed to evaluate LLMs in realistic and hard tasks through interaction with real-world MCP servers.<n>Our benchmark encompasses 6 core domains spanning 11 different MCP servers: Location Navigation, Repository Management, Financial Analysis, 3D Design, Browser Automation, and Web Searching.<n>We find that even SOTA models such as GPT-5 (43.72%), Grok-4 (33.33%) and Claude-4.0-Sonnet (29.44%) exhibit significant performance limitations.
  arXiv  Detail & Related papers  (2025-08-20T13:28:58Z)
• Compromising Embodied Agents with Contextual Backdoor Attacks [69.71630408822767]
  Large language models (LLMs) have transformed the development of embodied intelligence. This paper uncovers a significant backdoor security threat within this process. By poisoning just a few contextual demonstrations, attackers can covertly compromise the contextual environment of a black-box LLM.
  arXiv  Detail & Related papers  (2024-08-06T01:20:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.