MCP service LLM

LLM API Boilerplate

用于构建具有LLM（大型语言模型）集成的应用程序的NestJS样板。为多个LLM提供程序提供一个干净的抽象层，支持工具/函数调用。

特性

• 🔌 提供者抽象 -轻松在Gemini、OpenAI之间切换
• 🛠️ 工具/函数调用 -具有动态注册功能的可扩展工具注册表
• 🔄 递归工具循环 -自动处理多回转刀具对话
• 📡 实时聊天 -用于流式交互的WebSocket网关
• 🌐 简单HTTP API -只需发送 sessionId + message
• 💾 会话缓存 -内存历史管理（数据库就绪）
• 🧩 模块化设计 -易于扩展和定制

快速开始

1.安装依赖项

cd llm-api-boilerplate
npm install

2.配置环境

cp .env.example .env

编辑 .env 并添加您的API密钥：

GEMINI_API_KEY=your_gemini_api_key
OPENAI_API_KEY=your_openai_api_key  # Optional
PORT=3000

3.启动服务器

npm run start:dev

4.测试

HTTP（简化-只有会话ID和消息！）：

curl -X POST http://localhost:3000/chat/message \
  -H "Content-Type: application/json" \
  -d '{
    "sessionId": "user-123",
    "message": "What time is it?"
  }'

答复：

{
  "success": true,
  "sessionId": "user-123",
  "response": {
    "text": "The current time is 10:30 AM UTC.",
    "toolCalls": []
  }
}

Websocket：

const socket = io('http://localhost:3000/chat', {
  query: { sessionId: 'user-123' }
});

socket.on('ai_message', (data) => console.log('AI:', data.text));
socket.emit('message', { text: 'Hello!' });

会话管理：

# Get session info
curl http://localhost:3000/chat/session/user-123

# Delete session (clear history)
curl -X DELETE http://localhost:3000/chat/session/user-123

______________________________________________________________________

项目结构

src/
├── libs/
│   └── llm/
│       ├── interfaces/
│       │   ├── llm.types.ts           # Provider-agnostic types
│       │   └── chat-history.types.ts  # History manager interface
│       ├── history/
│       │   ├── gemini-history.manager.ts  # Gemini history format
│       │   ├── openai-history.manager.ts  # OpenAI history format
│       │   └── index.ts
│       └── providers/
│           ├── base-llm.provider.ts   # Abstract base class
│           ├── gemini.provider.ts     # Google Gemini
│           └── openai.provider.ts     # OpenAI GPT
├── modules/
│   ├── llm/
│   │   ├── services/
│   │   │   ├── llm-factory.service.ts    # Provider factory
│   │   │   ├── tool-handler.service.ts   # Tool registry & execution
│   │   │   └── orchestrator.service.ts   # Conversation loop
│   │   └── llm.module.ts
│   └── chat/
│       ├── chat.gateway.ts     # WebSocket handler
│       ├── chat.controller.ts  # HTTP endpoints
│       └── chat.module.ts
├── app.module.ts
└── main.ts

______________________________________________________________________

提供商感知历史管理

不同的LLM提供者处理工具调用的方式不同。此样板包括 特定于提供商的历史管理器 正确格式化工具调用和结果。

为何这很重要

双子座 （以下 官方文件):

// Tool results are sent as USER role with functionResponse
contents.push(response.candidates[0].content);  // Model's tool call
contents.push({ role: 'user', parts: [{ functionResponse: { name, response } }] });

开放人工智能:

// Tool results are sent as TOOL role with tool_call_id
messages.push({ role: 'assistant', tool_calls: [...] });  // Model's tool call
messages.push({ role: 'tool', tool_call_id: '...', content: '...' });

直接使用历史管理器

import { getHistoryManager } from 'src/libs/llm/history';

const historyManager = getHistoryManager('gemini'); // or 'openai'

// Build history in provider-native format
let history = [];
history = historyManager.addUserMessage(history, 'Hello');
history = historyManager.addAssistantToolCalls(history, toolCalls, rawResponse);
history = historyManager.addToolResults(history, toolCall, result);

// Use with provider
const response = await provider.generateWithNativeHistory(history, systemPrompt, tools);

______________________________________________________________________

注册自定义工具

工具允许LLM执行操作。在您的服务中注册它们：

import { Injectable, OnModuleInit } from '@nestjs/common';
import { ToolHandlerService } from './modules/llm';

@Injectable()
export class MyToolsService implements OnModuleInit {
  constructor(private toolHandler: ToolHandlerService) {}

  onModuleInit() {
    // Register a weather tool
    this.toolHandler.registerTool(
      {
        name: 'get_weather',
        description: 'Get current weather for a location',
        parameters: {
          type: 'object',
          properties: {
            location: { type: 'string', description: 'City name' }
          },
          required: ['location']
        }
      },
      async (toolCall, context) => {
        const { location } = toolCall.args;
        // Call your weather API here
        return {
          success: true,
          message: `Weather for ${location}: Sunny, 25°C`,
          data: { location, temp: 25, condition: 'sunny' }
        };
      }
    );
  }
}

______________________________________________________________________

添加新的LLM提供程序

1. 扩展 BaseLLMProvider:

// src/libs/llm/providers/anthropic.provider.ts
import { BaseLLMProvider } from './base-llm.provider';
import { LLMMessage, LLMResponse, ToolDefinition } from '../interfaces/llm.types';

export class AnthropicProvider extends BaseLLMProvider {
  async generateResponse(
    messages: LLMMessage[],
    systemPrompt?: string,
    tools?: ToolDefinition[],
  ): Promise {
    // Implement Claude API call here
  }
}

1. 注册于 LLMFactoryService:

case 'anthropic': {
  const apiKey = this.configService.get('ANTHROPIC_API_KEY');
  return new AnthropicProvider(providerConfig, apiKey);
}

______________________________________________________________________

MCP集成

要连接到MCP（模型上下文协议）服务器，请执行以下操作：

1. 创建MCP客户端服务
2. 通过以下方式动态注册MCP工具 ToolHandlerService
3. 将工具调用转发到MCP服务器

示例结构：

@Injectable()
export class MCPClientService implements OnModuleInit {
  constructor(private toolHandler: ToolHandlerService) {}

  async onModuleInit() {
    // Connect to MCP server
    const mcpTools = await this.discoverMCPTools();
    
    // Register each MCP tool
    for (const tool of mcpTools) {
      this.toolHandler.registerTool(tool.definition, async (call, ctx) => {
        return this.executeOnMCPServer(tool.name, call.args);
      });
    }
  }
}

______________________________________________________________________

API 参考

HTTP端点

POST/聊天/消息

发送消息并获得响应。

请求：

{
  "message": "Hello!",
  "sessionId": "optional-session-id",
  "llmConfig": {
    "provider": "gemini",
    "model": "gemini-2.0-flash-exp",
    "parameters": {
      "temperature": 0.7,
      "maxTokens": 2048
    }
  },
  "systemPrompt": "You are a helpful assistant.",
  "history": []
}

答复：

{
  "success": true,
  "sessionId": "session-123",
  "response": {
    "text": "Hello! How can I help you today?",
    "toolCalls": []
  },
  "usage": {
    "inputTokens": 10,
    "outputTokens": 15,
    "totalTokens": 25
  }
}

WebSocket事件

命名空间： /chat

______________________________________________________________________

许可证

麻省理工学院