URL Fetch MCP

URL获取MCP

一个干净的模型上下文协议（MCP）实现，使Claude或任何LLM能够从URL中获取内容。

特性

• 从任何URL获取内容
• 支持多种内容类型（HTML、JSON、文本、图像）
• 控制请求参数（标头、超时）
• 干净的错误处理
• 适用于Claude Code和Claude Desktop

存储库结构

url-fetch-mcp/
├── examples/               # Example scripts and usage demos
├── scripts/                # Helper scripts (installation, etc.)
├── src/
│   └── url_fetch_mcp/      # Main package code
│       ├── __init__.py
│       ├── __main__.py
│       ├── cli.py          # Command-line interface
│       ├── fetch.py        # URL fetching utilities
│       ├── main.py         # Core MCP server implementation
│       └── utils.py        # Helper utilities
├── LICENSE
├── pyproject.toml          # Project configuration
├── README.md
└── url_fetcher.py          # Standalone launcher for Claude Desktop

安装

# Install from source
pip install -e .

# Install with development dependencies
pip install -e ".[dev]"

用法

运行服务器

# Run with stdio transport (for Claude Code)
python -m url_fetch_mcp run

# Run with HTTP+SSE transport (for remote connections)
python -m url_fetch_mcp run --transport sse --port 8000

在Claude Desktop中安装

在Claude Desktop中安装有三种方法：

方法1：直接安装

# Install the package
pip install -e .

# Install in Claude Desktop using the included script
mcp install url_fetcher.py -n "URL Fetcher"

这 url_fetcher.py 文件包含：

#!/usr/bin/env python
"""
URL Fetcher MCP Server

This is a standalone script for launching the URL Fetch MCP server.
It's used for installing in Claude Desktop with the command:
    mcp install url_fetcher.py -n "URL Fetcher"
"""

from url_fetch_mcp.main import app

if __name__ == "__main__":
    app.run()

方法2：使用安装程序脚本

# Install the package
pip install -e .

# Run the installer script
python scripts/install_desktop.py

这 scripts/install_desktop.py 脚本：

#!/usr/bin/env python
import os
import sys
import tempfile
import subprocess

def install_desktop():
    """Install URL Fetch MCP in Claude Desktop."""
    print("Installing URL Fetch MCP in Claude Desktop...")
    
    # Create a temporary Python file that imports our module
    temp_dir = tempfile.mkdtemp()
    temp_file = os.path.join(temp_dir, "url_fetcher.py")
    
    with open(temp_file, "w") as f:
        f.write("""#!/usr/bin/env python
# URL Fetcher MCP Server
from url_fetch_mcp.main import app

if __name__ == "__main__":
    app.run()
""")
    
    # Make the file executable
    os.chmod(temp_file, 0o755)
    
    # Run the mcp install command with the file path
    try:
        cmd = ["mcp", "install", temp_file, "-n", "URL Fetcher"]
        print(f"Running: {' '.join(cmd)}")
        
        result = subprocess.run(cmd, check=True, text=True)
        print("Installation successful!")
        print("You can now use the URL Fetcher tool in Claude Desktop.")
        return 0
    except subprocess.CalledProcessError as e:
        print(f"Error during installation: {str(e)}")
        return 1
    finally:
        # Clean up temporary file
        try:
            os.unlink(temp_file)
            os.rmdir(temp_dir)
        except:
            pass

if __name__ == "__main__":
    sys.exit(install_desktop())

方法3：使用CLI命令

# Install the package
pip install -e .

# Install using the built-in CLI command
python -m url_fetch_mcp install-desktop

核心实施

MCP的主要实现在 src/url_fetch_mcp/main.py:

from typing import Annotated, Dict, Optional
import base64
import json
import httpx
from pydantic import AnyUrl, Field
from mcp.server.fastmcp import FastMCP, Context

# Create the MCP server
app = FastMCP(
    name="URL Fetcher",
    version="0.1.0",
    description="A clean MCP implementation for fetching content from URLs",
)

@app.tool()
async def fetch_url(
    url: Annotated[AnyUrl, Field(description="The URL to fetch")],
    headers: Annotated[
        Optional[Dict[str, str]], Field(description="Additional headers to send with the request")
    ] = None,
    timeout: Annotated[int, Field(description="Request timeout in seconds")] = 10,
    ctx: Context = None,
) -> str:
    """Fetch content from a URL and return it as text."""
    # Implementation details...

@app.tool()
async def fetch_image(
    url: Annotated[AnyUrl, Field(description="The URL to fetch the image from")],
    timeout: Annotated[int, Field(description="Request timeout in seconds")] = 10,
    ctx: Context = None,
) -> Dict:
    """Fetch an image from a URL and return it as an image."""
    # Implementation details...

@app.tool()
async def fetch_json(
    url: Annotated[AnyUrl, Field(description="The URL to fetch JSON from")],
    headers: Annotated[
        Optional[Dict[str, str]], Field(description="Additional headers to send with the request")
    ] = None,
    timeout: Annotated[int, Field(description="Request timeout in seconds")] = 10,
    ctx: Context = None,
) -> str:
    """Fetch JSON from a URL, parse it, and return it formatted."""
    # Implementation details...

工具功能

fetch_url

从URL获取内容并将其作为文本返回。

参数：

• url （必填）：要获取的URL
• headers （可选）：与请求一起发送的其他标头
• timeout （可选）：请求超时时间（秒）（默认值：10）

fetch_image

从URL获取图像并将其作为图像返回。

参数：

• url （必填）：从中获取图像的URL
• timeout （可选）：请求超时时间（秒）（默认值：10）

fetch_json

从URL获取JSON，解析它，并返回格式化的JSON。

参数：

• url （必填）：从中获取JSON的URL
• headers （可选）：与请求一起发送的其他标头
• timeout （可选）：请求超时时间（秒）（默认值：10）

例子

这 examples 目录包含示例脚本：

• quick_test.py：MCP服务器的快速测试
• simple_usage.py：使用客户端API的示例
• interactive_client.py：用于测试的交互式CLI

# Example of fetching a URL
result = await session.call_tool("fetch_url", {
    "url": "https://example.com"
})

# Example of fetching JSON data
result = await session.call_tool("fetch_json", {
    "url": "https://api.example.com/data",
    "headers": {"Authorization": "Bearer token"}
})

# Example of fetching an image
result = await session.call_tool("fetch_image", {
    "url": "https://example.com/image.jpg"
})

测试

要测试基本功能：

# Run a direct test of URL fetching
python direct_test.py

# Run a simplified test with the MCP server
python examples/quick_test.py

许可证

麻省理工学院