IndexFoundry MCP

IndexFoundry MCP

确定性向量索引工厂 -一个MCP服务器，用于从任何内容源自动创建可审计的矢量数据库，并具有可部署的基于项目的工作流。

工具不会思考，它们会行动。

此服务器中的每个工具都是：

• 确定性的：相同的输入→ 相同的输出
• 幂等：重新运行会产生相同的工件（除非 force: true)
• 可审计的：每个操作都会生成清单、哈希和日志
• 可组合：工具可以独立运行或链接运行

建筑

IndexFoundry提供两个互补的工作流程：

1.基于运行的管道（细粒度控制）

单个管道使用隔离的工件运行，适用于实验和详细审计。

2.基于项目的工作流（可部署的RAG应用程序）

使用MCP服务器、Dockerfile和Railway配置生成部署就绪存储库的自包含项目。

管道阶段（基于运行）

Connect → Extract → Normalize → Index → Serve
   ↓         ↓          ↓          ↓       ↓
  raw/    extracted/  normalized/  indexed/  served/

第一阶段：连接

从URL、网站地图、文件夹或PDF中获取内容。每个工件都有一个内容哈希。

第二阶段：提取

使用固定提取器（pdfminer、cheerio等）将原始字节转换为文本。

第三阶段：正常化

确定性地分块文本，丰富元数据（无LLM），并消除重复。

第四阶段：索引

使用固定模型生成嵌入，并上传到向量数据库。

第五阶段：发球

生成OpenAPI规范，并可选地启动检索API。

快速开始

# Install dependencies
npm install

# Build
npm run build

# Run on stdio (for Claude Desktop, Cline, etc.)
npm start

# Run as HTTP server
npm run start:http

工作流选项

选项1：基于运行的管道（详细控制）

使用单独的管道工具对每个阶段进行细粒度控制：

// Create a new run
const runId = crypto.randomUUID();
await client.callTool("indexfoundry_connect_folder", {
  run_id: runId,
  path: "/path/to/documents",
  glob: "**/*.pdf"
});

// Extract PDF content
await client.callTool("indexfoundry_extract_pdf", {
  run_id: runId,
  pdf_path: "raw/.pdf",
  mode: "layout"
});

// Chunk: text
await client.callTool("indexfoundry_normalize_chunk", {
  run_id: runId,
  input_paths: ["extracted/.pages.jsonl"],
  strategy: "recursive",
  max_chars: 1500,
  overlap_chars: 150
});

// Generate embeddings
await client.callTool("indexfoundry_index_embed", {
  run_id: runId,
  model: {
    provider: "openai",
    model_name: "text-embedding-3-small",
    api_key_env: "OPENAI_API_KEY"
  }
});

// Upsert to vector DB
await client.callTool("indexfoundry_index_upsert", {
  run_id: runId,
  provider: "local",
  connection: { collection: "my_docs" }
});

选项2：基于项目的工作流（可部署RAG）

创建一个自包含、可部署的RAG应用程序：

// Create a new project
await client.callTool("indexfoundry_project_create", {
  project_id: "my-rag-app",
  name: "My RAG Search",
  description: "Searchable knowledge base for documentation",
  embedding_model: {
    provider: "openai",
    model_name: "text-embedding-3-small",
    api_key_env: "OPENAI_API_KEY"
  },
  chunk_config: {
    strategy: "recursive",
    max_chars: 1500,
    overlap_chars: 150
  }
});

// Add data sources
await client.callTool("indexfoundry_project_add_source", {
  project_id: "my-rag-app",
  url: "https://docs.example.com",
  source_name: "Documentation Site",
  tags: ["docs", "api"]
});

// Build: vector database
await client.callTool("indexfoundry_project_build", {
  project_id: "my-rag-app"
});

// Query: built index
await client.callTool("indexfoundry_project_query", {
  project_id: "my-rag-app",
  query: "How do I configure authentication?",
  mode: "hybrid",
  top_k: 5
});

// Export for deployment
await client.callTool("indexfoundry_project_export", {
  project_id: "my-rag-app",
  server_name: "my-rag-server",
  include_http: true,
  railway_config: true
});

导出后，项目目录包含一个完整的可部署存储库：

• Dockerfile -容器配置
• railway.toml -铁路部署配置
• src/index.ts -使用搜索工具生成MCP服务器
• README.md -项目特定文件

推送到GitHub并部署：

cd projects/my-rag-app
git init
git add .
git commit -m "Initial RAG application"
git push
# Then connect to Railway and deploy

工具概述

基于运行的管道工具

连接阶段

• indexfoundry_connect_url -获取具有域分配的单个URL
• indexfoundry_connect_sitemap -使用URL过滤抓取站点地图
• indexfoundry_connect_folder -加载具有glob模式的本地文件
• indexfoundry_connect_pdf -使用元数据提取获取PDF

萃取相

• indexfoundry_extract_pdf -PDF转文本（布局/纯文本/OCR模式）
• indexfoundry_extract_html -HTML用于清理文本并保留结构
• indexfoundry_extract_document -通用文档提取（markdown、txt、CSV、JSON）

标准化阶段

• indexfoundry_normalize_chunk -将文本分割成块（递归/段落/标题/页面/句子/固定）
• indexfoundry_normalize_enrich -添加元数据（语言检测、正则表达式标签、节分类）
• indexfoundry_normalize_dedupe -删除重复项（精确/simhash/minhash）

指数阶段

• indexfoundry_index_embed -生成嵌入（OpenAI/Cohere/句子转换器/本地）
• indexfoundry_index_upsert -写入向量DB（松果体/织物/Qdrant/Milvus/Chroma/局部）
• indexfoundry_index_build_profile -配置检索（top_k、混合搜索、重新排序）

服务阶段

• indexfoundry_serve_openapi -生成OpenAPI 3.1规范
• indexfoundry_serve_start -启动HTTP搜索API服务器
• indexfoundry_serve_stop -停止运行API服务器
• indexfoundry_serve_status -获取服务器状态
• indexfoundry_serve_query -直接查询运行服务器

运行实用程序

• indexfoundry_run_status -获取跑步的详细状态
• indexfoundry_run_list -列出所有经过筛选的跑步记录
• indexfoundry_run_diff -比较两次运行（配置、块、时间）
• indexfoundry_run_cleanup -使用保留策略删除旧运行

基于项目的工作流工具

项目管理

• indexfoundry_project_create -使用嵌入和块配置创建新项目
• indexfoundry_project_list -列出所有具有可选统计信息的项目
• indexfoundry_project_get -获取项目详细信息、清单和来源
• indexfoundry_project_delete -删除项目（需要 confirm: true)

来源管理

• indexfoundry_project_add_source -添加带有标签的数据源（url/sitemap/文件夹/pdf）

构建和查询

• indexfoundry_project_build -处理所有挂起的源（获取、块、嵌入、追加销售）
• indexfoundry_project_query -搜索项目的矢量数据库（语义/关键字/混合）

部署

• indexfoundry_project_export -生成部署文件（Dockerfile、MCP服务器、railway.toml）

目录结构

基于运行的结构

runs//
├── manifest.json           # Master audit trail
├── config.json             # Frozen config
├── raw/                    # Fetched artifacts
├── extracted/              # Text extraction
├── normalized/             # Chunks
├── indexed/                # Embeddings
├── served/                 # API artifacts
└── logs/                   # Event logs

基于项目的结构

projects/
/
├── project.json            # Project manifest (embedding config, stats)
├── sources.jsonl          # Source records (url/sitemap/folder/pdf)
├── data/
│   ├── chunks.jsonl       # Indexed chunks
│   └── vectors.jsonl     # Generated embeddings
├── runs/                  # Per-source build runs
├── src/
│   └── index.ts         # Generated MCP server
├── Dockerfile             # Container configuration
├── railway.toml           # Railway deployment config
├── package.json           # Server dependencies
├── tsconfig.json          # TypeScript config
└── README.md             # Project documentation

配置

环境变量

# Run-based pipeline
INDEXFOUNDRY_RUNS_DIR=./runs     # Where to store runs

# Embeddings
OPENAI_API_KEY=sk-...           # For OpenAI embeddings
EMBEDDING_API_KEY=sk-...         # Generic env variable (configurable per project)

# Server
PORT=3000                        # For HTTP transport
TRANSPORT=stdio                  # stdio or http

项目配置

项目存储配置 project.json:

{
  "project_id": "my-rag",
  "name": "My RAG Search",
  "embedding_model": {
    "provider": "openai",
    "model_name": "text-embedding-3-small",
    "api_key_env": "OPENAI_API_KEY"
  },
  "chunk_config": {
    "strategy": "recursive",
    "max_chars": 1500,
    "overlap_chars": 150
  }
}

示例用法

基于运行的管道示例

// Create a new run
const runId = crypto.randomUUID();

// Connect: fetch from folder
await client.callTool("indexfoundry_connect_folder", {
  run_id: runId,
  path: "/path/to/documents",
  glob: "**/*.pdf"
});

// Extract: PDF to text
await client.callTool("indexfoundry_extract_pdf", {
  run_id: runId,
  pdf_path: "raw/.pdf",
  mode: "layout"
});

// Normalize: chunk text
await client.callTool("indexfoundry_normalize_chunk", {
  run_id: runId,
  input_paths: ["extracted/.pages.jsonl"],
  strategy: "recursive",
  max_chars: 1500,
  overlap_chars: 150
});

// Index: generate embeddings
await client.callTool("indexfoundry_index_embed", {
  run_id: runId,
  model: {
    provider: "openai",
    model_name: "text-embedding-3-small",
    api_key_env: "OPENAI_API_KEY"
  }
});

// Upsert to local vector DB
await client.callTool("indexfoundry_index_upsert", {
  run_id: runId,
  provider: "local",
  connection: { collection: "my_docs" }
});

// Serve: start HTTP API
await client.callTool("indexfoundry_serve_start", {
  run_id: runId,
  port: 8080
});

基于项目的工作流示例

// Create a deployable RAG project
await client.callTool("indexfoundry_project_create", {
  project_id: "my-docs-rag",
  name: "Company Documentation Search",
  description: "Searchable knowledge base for internal docs",
  embedding_model: {
    provider: "openai",
    model_name: "text-embedding-3-small",
    api_key_env": "OPENAI_API_KEY"
  },
  chunk_config: {
    strategy: "recursive",
    max_chars: 1500,
    overlap_chars: 150
  }
});

// Add multiple sources
await client.callTool("indexfoundry_project_add_source", {
  project_id: "my-docs-rag",
  url: "https://docs.company.com",
  source_name: "Main Docs",
  tags: ["docs", "internal"]
});

await client.callTool("indexfoundry_project_add_source", {
  project_id: "my-docs-rag",
  folder_path: "/path/to/pdfs",
  source_name: "Policy Documents",
  tags: ["policy", "pdf"]
});

// Build: vector database
await client.callTool("indexfoundry_project_build", {
  project_id: "my-docs-rag"
});

// Query: index
const results = await client.callTool("indexfoundry_project_query", {
  project_id: "my-docs-rag",
  query: "What is the vacation policy?",
  mode: "hybrid",
  top_k: 5,
  filter_tags: ["policy"]
});

// Export for deployment
await client.callTool("indexfoundry_project_export", {
  project_id: "my-docs-rag",
  server_name: "docs-search-server",
  server_description: "Internal documentation search API",
  include_http: true,
  railway_config: true
});

导出后，项目目录包含一个可部署的存储库：

cd projects/my-docs-rag
git init
git add .
git commit -m "Initial RAG application"
git push origin main
# Deploy on Railway

发展

# Development with watch mode
npm run dev

# Run tests (single run)
npm test

# Run tests (watch mode)
npm run test:watch

# Lint
npm run lint

# Test with MCP Inspector
npm run inspector

测试

MCP服务器已通过端到端测试进行了验证：

• ✅ 项目创建、列出和检索
• ✅ 源代码添加（URL、文件夹、PDF、站点地图）
• ✅ 构建管道（获取→ 块→ 嵌入→ 扰乱）
• ✅ 具有语义、关键字和混合模式的矢量搜索
• ✅ 部署文件生成（Dockerfile、railway.toml、MCP服务器）

部署

铁路部署

1. 创建和导出项目：

await client.callTool("indexfoundry_project_export", {
  project_id: "my-rag",
  railway_config: true
});

1. 推送到GitHub并连接到Railway

1. 铁路自动检测 railway.toml 并部署

Docker部署

cd projects/my-rag
docker build -t my-rag-server .
docker run -p 8080:8080 -e OPENAI_API_KEY=sk-... my-rag-server

决定论保证

1. 排序输入：处理前排序的文件列表
2. 稳定ID：从内容+位置导出的块ID
3. 内容哈希：每个工件上都有SHA256
4. 固定版本：提取器版本锁定在配置中
5. 无随机性：无采样、混洗或非确定性算法

许可证

专有软件许可证