MCP Discovery

mcp发现

LLM驱动的推理，具有本地MCP工具发现和执行功能。

安装

需要 包子 运行时。

bunx mcp-discovery

无需安装- bunx 将自动下载并运行最新版本。

用法

MCP Discovery - LLM inference with local MCP tools

Executes LLM inference calls with automatic tool discovery from MCP servers.
Supports multiple providers and tool discovery strategies.

Requirements:
  - Bun runtime (https://bun.sh)
  - Environment variables for provider (e.g., GROQ_API_KEY)
  - MCP servers configured in mcp/servers.json

Usage:
  Single run:
    bunx mcp-discovery  -m 
 [-s ] -p "
"

  Benchmark mode:
    bunx mcp-discovery  -m 
 [-s ] -p "
" -n  [-c ]

Arguments:
             Tool discovery strategy (currently: 'all')
                       - all: Include all tools from all servers
                       - minimal: Use VCR cache to find minimal set of tools for task
                       - all-relevant: Use only tools from servers specified in prompts file
                       
                       Suffix with :strict to enable strict tool definition mode
                       Example: all:strict, minimal:strict

  -m 
  Provider and model specification
                       Format: provider:model
                       Examples: groq:llama-3.1-70b-versatile
                                 openai:gpt-4o-mini

  -s          [Optional] Comma-separated list of MCP server IDs
                       Available: ppt, playwright, word, chart, trends
                       If not provided, runs without tools

  -p 
          User prompt to send to the LLM

  -n             [Optional] Number of times to run (default: 1)
                       If > 1, enters benchmark mode with statistics

  -c      [Optional] Number of concurrent workers (default: 1)
                       Each worker maintains its own MCP server pool

  -x      [Optional] Expected text in  block (multiple allowed)
                       When provided, pass/fail based on whether all expectations found
                       Text is normalized (case-insensitive, normalized punctuation)

  --help               Show this help message

Modes:
  Single run (n=1):    Outputs progress then complete JSON response

  Benchmark (n>1):     Runs multiple inferences concurrently
                       Shows progress, statistics, and latency metrics
                       Saves per-run logs to temp directory
                       Returns exit code 0 if all pass, 1 if any fail

Providers:
  Configure in providers.json. Currently supported:
  - groq (adapter: completions)
  - openai (adapter: completions, stub)
  - openrouter (adapter: completions, stub)
  - anthropic (adapter: messages, stub)
  - groq-responses (adapter: responses, stub)

Examples:
  # Single run - simple query
  bunx mcp-discovery all \
    -m groq:llama-3.3-70b-versatile \
    -p "What is 2+2?"

  # Single run - query bundled PowerPoint file (relative path)
  bunx mcp-discovery all \
    -m groq:llama-3.3-70b-versatile \
    -s ppt \
    -p "What is the title of data/ppt/build_effective_agents.pptx?"

  # Benchmark - test reliability with 50 runs
  bunx mcp-discovery all \
    -m groq:llama-3.3-70b-versatile \
    -s ppt \
    -p "What is the title of data/ppt/build_effective_agents.pptx?" \
    -n 50 -c 8

  # Benchmark - high concurrency stress test
  bunx mcp-discovery all \
    -m groq:llama-3.1-8b-instant \
    -p "What is 5+5?" \
    -n 100 -c 10

  # Benchmark with expectations (checks for specific text in answer)
  bunx mcp-discovery all \
    -m groq:llama-3.3-70b-versatile \
    -p "What is 2 + 2. Put only the answer between  tags." \
    -n 20 -c 4 \
    -x "4"

  # Run with "all-relevant" strategy (requires prompts file)
  bunx mcp-discovery all-relevant \
    -m groq:llama-3.3-70b-versatile \
    -i mcp/prompts.ts:0

  # Extract just the content from single run (filter out progress lines)
  bunx mcp-discovery all \
    -m groq:llama-3.3-70b-versatile \
    -p "Hello" \
    | tail -1 | jq -r '.choices[0].message.content'

配置

提供商

编辑\providers.json\配置LLM提供程序。每个提供者指定：

• adapter：API类型（完成、响应、消息）
• baseURL：API端点
• apiKeyEnv：API键的环境变量
• defaultModel：默认型号（如果未指定）
• stub：如果为true，则提供程序尚未实现

MCP服务器

编辑\mcp/servers.json\配置MCP服务器。每个服务器指定：

• id：唯一标识符
• command：可执行命令
• args：命令参数
• env：环境变量（可选）

MCP服务器会自动启动，其工作目录设置为 mcp/，所以相对路径如下 data/ppt/file.pptx 正确工作。

测试数据

示例数据文件包含在\[mcp/data/\]中(file:///Users/glenmaddern/src/projects/mcp-discovery/mcp/data/)用于测试：

• ppt/ -PowerPoint演示文稿
• word/ -Word文档

这些文件来自 LiveMCPBench 带注释的数据集。

建筑

src/
├── types/           Type definitions
├── config/          Configuration loaders
├── adapters/        LLM provider adapters
│   ├── completions/ OpenAI-style completions API
│   ├── responses/   [Stub] Responses API
│   └── messages/    [Stub] Messages API (Anthropic)
├── strategies/      Tool discovery strategies
│   └── discovery/
│       ├── all.ts   Include all tools
│       ├── minimal.ts Use VCR cache for minimal set
│       ├── all-relevant.ts Filter by relevant servers
│       ├── browse   [Stub] Interactive browsing
│       └── search   [Stub] Search-based discovery
├── mcp/            MCP client implementation
└── core/           Core tool execution loop

日志分析

包含一个独立的实用程序，用于分析基准运行的日志。它生成一个颜色编码的报告，按提示和错误类型对失败进行分组。

# Analyze a specific log directory
bun src/stats.ts /var/folders/xx/yyyy/T/mcp_discovery_run_id

特征：

• 按提示分组失败
• 提取未达到预期的返回答案
• 识别系统错误（JSON解析、工具验证）
• 计算每个提示的成功率
• 支持传统和基于元数据的日志格式

贡献

在本地开发此项目：

# Clone the repository
git clone 
cd mcp-discovery

# Install dependencies
bun install

# Run locally during development
bun src/mcp-discovery.ts all -m groq:llama-3.3-70b-versatile -p "Test"

# The CLI entrypoint is src/mcp-discovery.ts
# All other source code is in the src/ directory

项目结构

• src/mcp-discovery.ts -CLI入口点
• src/ -源代码模块
• providers.json -LLM提供程序配置
• mcp/servers.json -MCP服务器配置

开发工作流程

1. 对源文件进行更改
2. 测试用 bun src/mcp-discovery.ts
3. 用描述性消息提交更改
4. 所有输出都进入stdout

许可证

麻省理工学院