Robot Resources Scraper
MCP Server
Web scraper and token compressor that converts HTML to clean markdown with 70-80% fewer tokens. Single-page compression and multi-page BFS crawling with auto-fallback fetch modes.
工具数
快速接入
先看主来源和安装命令，再打开仓库或文档；下面只保留这个条目的关键接入事实。
目录标签
目录标签
接入字段
传输方式（transport，传输协议）
未说明
鉴权方式（authType，认证方式）
none
部署方式（deploymentType，部署类型）
local-only
权限和风险
接入前请确认传输方式、认证方式和部署位置，并根据实际工具能力限制访问范围。
不要直接授予不必要的文件、网络或账号权限；先核对安装命令和配置内容。
仍需确认：category / installCommand
来源信息
继续浏览同类 MCP
上一个 MCP
robotdomainsearch
RobotDomainSearch is a domain intelligence API built for AI agents. Check domain availability across 512+ TLDs, get AI-powered valuations with comparable sales data, run WHOIS/RDAP lookups, browse aftermarket listings and auctions, check ENS names, and analyze online name presence — all through a single MCP server. Supports x402 payments for premium APIs.
下一个 MCP
Robotstxt AI MCP
Analyze and generate robots.txt files with AI crawler awareness. Fetch any site's robots.txt, detect which AI bots (GPTBot, ClaudeBot, PerplexityBot, Google-Extended) are blocked or allowed, and generate optimized robots.txt with toggle controls for 20+ AI crawlers.