An Empirical Study of Model Context Protocol Applications

• URL: http://arxiv.org/abs/2607.25635v2
• Date: Wed, 29 Jul 2026 11:36:56 GMT
• Title: An Empirical Study of Model Context Protocol Applications
• Abstract summary: The Model Context Protocol (MCP) standardizes how large language model applications communicate with external tools, but leaves the application side unspecified.<n>This ecosystem is also under-researched, with existing work focused on servers rather than the applications consuming them.<n>We conduct a large-scale study of 1,723 MCPApps mined from GitHub.
• Score: 0.9379969114114787
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The Model Context Protocol (MCP) standardizes how large language model applications communicate with external tools, but leaves the application side unspecified: unlike traditional dependencies resolved through package managers, developers integrating MCP servers face no conventions for configuration, communication, or human oversight. This ecosystem is also under-researched, with existing work focused on servers rather than the applications consuming them. We conduct a large-scale study of 1,723 MCPApps mined from GitHub. We first derive MCPAppTax from a representative sample, then use an LLM-assisted pipeline to apply it across the full dataset, characterizing server integration across configuration, SDK use, and human-in-the-loop mechanisms. Our results show that the ecosystem has converged on some practices but not others: most MCPApps configure servers using files (85.2%) and use an official SDK (81.1%) to communicate with servers, yet no naming convention has emerged for configuration files. Human oversight diverges most, logging (90.8%) and enable/disable controls (77.2%) are common, but only 37.2% gate tool execution behind a blocking approval step, leaving the LLM able to invoke any enabled tool unconditionally in most MCPApps.

Related papers

• MCP-Persona: Benchmarking LLM Agents on Real-World Personal Applications via Environment Simulation [55.14253918630382]
  We introduce MCP-Persona, the first benchmark specifically designed for evaluating agent performance on real-world, personalized MCP tools.<n> MCP-Persona encompasses a diverse set of widely-used applications, ranging from social media platforms like Reddit and Xiaohongshu (Rednote) to enterprise collaboration suites such as Lark (Feishu) and Slack.<n>Our experiments on various state-of-the-art (SOTA) agents demonstrate their significant struggles with personalized tool use, thereby highlighting the benchmark's crucial role in identifying and addressing these limitations.
  arXiv  Detail & Related papers  (2026-06-01T16:44:10Z)
• ProgramBench: Can Language Models Rebuild Programs From Scratch? [59.40748183470308]
  ProgramBench measures the ability of software engineering agents to develop software holisitically.<n>End-to-end behavioral tests are generated via agent-driven fuzzing.<n>Models favor monolithic, single-file implementations that diverge sharply from human-written code.
  arXiv  Detail & Related papers  (2026-05-05T09:17:02Z)
• Separating Intelligence from Execution: A Workflow Engine for the Model Context Protocol [0.0]
  We present the MCP Engine, a novel orchestration layer that decouples from execution (carrying it out)<n>The engine reduces per-execution token cost by over 99%, completes the full cluster graph--comprising 1,200+ nodes and 2,800+ relationships--in under 45 seconds.
  arXiv  Detail & Related papers  (2026-03-13T05:12:14Z)
• MCP-Flow: Facilitating LLM Agents to Master Real-World, Diverse and Scaling MCP Tools [58.5971352939562]
  Large Language Models increasingly rely on external tools to perform complex, realistic tasks.<n>Existing MCP research covers few servers, depends on costly manual curation, and lacks training support.<n>We introduce MCP-Flow, an automated web-agent-driven pipeline for large-scale server discovery, data synthesis, and model training.
  arXiv  Detail & Related papers  (2025-10-28T10:42:17Z)
• Code2MCP: Transforming Code Repositories into MCP Services [53.234097255779744]
  Model Context Protocol (MCP) aims to create a standard for how Large Language Models use tools.<n>We introduce Code2MCP, an agent-based framework that automatically transforms a GitHub repository into a functional MCP service.<n>We demonstrate that Code2MCP successfully transforms open-source computing libraries in scientific fields such as bioinformatics, mathematics, and fluid dynamics.
  arXiv  Detail & Related papers  (2025-09-07T06:13:25Z)
• LiveMCPBench: Can Agents Navigate an Ocean of MCP Tools? [50.60770039016318]
  We present LiveMCPBench, the first comprehensive benchmark for benchmarking Model Context Protocol (MCP) agents.<n>LiveMCPBench consists of 95 real-world tasks grounded in the MCP ecosystem.<n>Our evaluation covers 10 leading models, with the best-performing model reaching a 78.95% success rate.
  arXiv  Detail & Related papers  (2025-08-03T14:36:42Z)
• Making REST APIs Agent-Ready: From OpenAPI to MCP Servers for Tool-Augmented LLMs [0.0]
  We present AutoMCP, a compiler that generates MCP servers from OpenAPI 2.0/3.0 specifications.<n>We evaluate AutoMCP on 50 real-world APIs spanning 5,066 endpoints across over 10 domains.
  arXiv  Detail & Related papers  (2025-07-21T20:20:31Z)
• MCPWorld: A Unified Benchmarking Testbed for API, GUI, and Hybrid Computer Use Agents [14.736516215309768]
  We propose MCPWorld, the first automatic CUA testbed for API, GUI, and API-GUI hybrid agents.<n>A key principle of MCPWorld is the use of "white-box apps", i.e., those with source code availability and can be revised/re-compiled as needed.<n> MCPWorld includes 201 well curated and annotated user tasks, covering diversified use cases and difficulty levels.
  arXiv  Detail & Related papers  (2025-06-09T11:50:33Z)
• MCP Bridge: A Lightweight, LLM-Agnostic RESTful Proxy for Model Context Protocol Servers [0.5266869303483376]
  MCP Bridge is a lightweight proxy that connects to multiple MCP servers and exposes their capabilities through a unified API.<n>The system implements a risk-based execution model with three security levels standard execution, confirmation, and Docker isolation while maintaining backward compatibility with standard MCP clients.
  arXiv  Detail & Related papers  (2025-04-11T22:19:48Z)
• SOPBench: Evaluating Language Agents at Following Standard Operating Procedures and Constraints [59.645885492637845]
  SOPBench is an evaluation pipeline that transforms each service-specific SOP code program into a directed graph of executable functions.<n>Our approach transforms each service-specific SOP code program into a directed graph of executable functions and requires agents to call these functions based on natural language SOP descriptions.<n>We evaluate 18 leading models, and results show the task is challenging even for top-tier models.
  arXiv  Detail & Related papers  (2025-03-11T17:53:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.