Real Faults in Model Context Protocol (MCP) Software: a Comprehensive Taxonomy

• URL: http://arxiv.org/abs/2603.05637v1
• Date: Thu, 05 Mar 2026 19:47:26 GMT
• Title: Real Faults in Model Context Protocol (MCP) Software: a Comprehensive Taxonomy
• Abstract summary: We present the first large-scale taxonomy of faults in Model Context Protocol-based software systems.<n>These insights can inform the development of more robust, reliable, and secure AI-enabled software systems.
• Score: 5.6795075579540395
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The rapid adoption of foundation models has significantly expanded the capabilities of software systems, enabling them to perform complex language, reasoning, and interaction tasks that were previously difficult to automate. However, this progress has also introduced novel challenges that were largely absent in previous generations of software. In particular, the increasing integration of foundation models with external tools and resources raises new concerns regarding reliability, security, and robustness. The Model Context Protocol (MCP) has recently been proposed to standardize interactions between AI-based software systems, software tools, and external resources. Despite its growing adoption, there remains limited systematic understanding of real-world faults in MCP-based software systems. In this paper, we present the first large-scale taxonomy of faults in MCP servers, comprising five high-level fault categories derived from empirical evidence. To evaluate the completeness and generalizability of this taxonomy, we conduct a survey of MCP practitioners with diverse roles and experience levels. The results confirm that all identified fault categories occur in practice and reveal distinct characteristics that differentiate MCP-specific faults from non-MCP faults. Overall, this study provides actionable insights for researchers and practitioners by identifying the most error-prone and critical components of MCP-based systems. These insights can inform the development of more robust, reliable, and secure AI-enabled software systems that rely on MCP.

Related papers

• Understanding How Enterprises Adopt the Model Context Protocol for LLM-Driven Software Engineering [4.345394092402742]
  Large Language Models (LLMs) are increasingly used in AI-based software engineering, but their limitations in complex task execution and multi-tool coordination have driven growing interest in the Model Context Protocol (MCP)<n>Existing research has mainly focused on MCP's technical design, with limited empirical evidence on how it is adopted and used in enterprise practice.<n>To address this gap, we conducted semi-structured interviews with 20 practitioners from eight companies in the Internet and financial sectors.
  arXiv  Detail & Related papers  (2026-06-08T08:20:50Z)
• Don't believe everything you read: Understanding and Measuring MCP Behavior under Misleading Tool Descriptions [20.772501129444993]
  We study how mismatches between externally presented tool descriptions and underlying implementations shape the mental models and decision-making behavior of intelligent agents.<n>Our results show that while most servers are highly consistent, approximately 13% exhibit substantial mismatches that can enable undocumented privileged operations, hidden state mutations, or unauthorized financial actions.<n>Our findings demonstrate that description-code inconsistency is a concrete and prevalent attack surface in MCP-based AI agents, and motivate the need for systematic auditing and stronger transparency guarantees in future agent ecosystems.
  arXiv  Detail & Related papers  (2026-02-03T14:31:52Z)
• Agentic Software Issue Resolution with Large Language Models: A Survey [9.583478737157531]
  Software issue resolution aims to address real-world issues in software repositories based on natural language descriptions provided by users.<n>Large language models (LLMs) in reasoning and generative capabilities have made significant progress in automated software issue resolution.<n>Recently, LLM-based agentic systems have become mainstream for software issue resolution.
  arXiv  Detail & Related papers  (2025-12-24T08:05:10Z)
• A Comprehensive Survey on Benchmarks and Solutions in Software Engineering of LLM-Empowered Agentic System [56.40989626804489]
  This survey provides the first holistic analysis of Large Language Models-powered software engineering.<n>We review over 150 recent papers and propose a taxonomy along two key dimensions: (1) Solutions, categorized into prompt-based, fine-tuning-based, and agent-based paradigms, and (2) Benchmarks, including tasks such as code generation, translation, and repair.
  arXiv  Detail & Related papers  (2025-10-10T06:56:50Z)
• A Defect Classification Framework for AI-Based Software Systems (AI-ODC) [0.0]
  This paper proposes a framework inspired by the Orthogonal Defect Classification (ODC) paradigm.<n>The framework was adapted to accommodate the Data, Learning, and Thinking aspects of AI systems.
  arXiv  Detail & Related papers  (2025-08-25T11:15:31Z)
• MCP-Universe: Benchmarking Large Language Models with Real-World Model Context Protocol Servers [86.00932417210477]
  We introduce MCP-Universe, the first comprehensive benchmark specifically designed to evaluate LLMs in realistic and hard tasks through interaction with real-world MCP servers.<n>Our benchmark encompasses 6 core domains spanning 11 different MCP servers: Location Navigation, Repository Management, Financial Analysis, 3D Design, Browser Automation, and Web Searching.<n>We find that even SOTA models such as GPT-5 (43.72%), Grok-4 (33.33%) and Claude-4.0-Sonnet (29.44%) exhibit significant performance limitations.
  arXiv  Detail & Related papers  (2025-08-20T13:28:58Z)
• We Urgently Need Privilege Management in MCP: A Measurement of API Usage in MCP Ecosystems [28.59170303701817]
  We conduct the first large-scale empirical analysis of Model Context Protocol security risks.<n>We examine 2,562 real-world MCP applications spanning 23 functional categories.<n>We propose a detailed taxonomy of MCP resource access, quantify security-relevant API usage, and identify open challenges for building safer MCP ecosystems.
  arXiv  Detail & Related papers  (2025-07-05T03:39:30Z)
• Rethinking Machine Unlearning in Image Generation Models [59.697750585491264]
  CatIGMU is a novel hierarchical task categorization framework.<n>EvalIGMU is a comprehensive evaluation framework.<n>We construct DataIGM, a high-quality unlearning dataset.
  arXiv  Detail & Related papers  (2025-06-03T11:25:14Z)
• Survey of LLM Agent Communication with MCP: A Software Design Pattern Centric Review [0.9208007322096533]
  The study revisits well-established patterns, including Mediator, Observer, Publish-Subscribe, and Broker.<n>The article concludes by outlining open challenges, potential security risks, and promising directions for advancing robust, interoperable, and scalable multi-agent ecosystems.
  arXiv  Detail & Related papers  (2025-05-26T09:11:17Z)
• Towards Human-Guided, Data-Centric LLM Co-Pilots [53.35493881390917]
  CliMB-DC is a human-guided, data-centric framework for machine learning co-pilots.<n>It combines advanced data-centric tools with LLM-driven reasoning to enable robust, context-aware data processing.<n>We show how CliMB-DC can transform uncurated datasets into ML-ready formats.
  arXiv  Detail & Related papers  (2025-01-17T17:51:22Z)
• Coding for Intelligence from the Perspective of Category [66.14012258680992]
  Coding targets compressing and reconstructing data, and intelligence. Recent trends demonstrate the potential homogeneity of these two fields. We propose a novel problem of Coding for Intelligence from the category theory view.
  arXiv  Detail & Related papers  (2024-07-01T07:05:44Z)
• Technology Readiness Levels for Machine Learning Systems [107.56979560568232]
  Development and deployment of machine learning systems can be executed easily with modern tools, but the process is typically rushed and means-to-an-end. We have developed a proven systems engineering approach for machine learning development and deployment. Our "Machine Learning Technology Readiness Levels" framework defines a principled process to ensure robust, reliable, and responsible systems.
  arXiv  Detail & Related papers  (2021-01-11T15:54:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.