人工智能浏览器自动化API
智能网络自动化 -使用Gemini LLM+剧作家MCP
一项生产就绪的FastAPI服务，使用谷歌的Gemini AI智能自动化网络搜索和自动化。代理可以导航、搜索、滚动和提取信息，无需人工干预。
______________________________________________________________________
特性
- 亚马逊优化：专门为Amazon.com工作流程设计的提示
- 智能产品搜索：自动搜索和查找产品
- 价格提取：从搜索结果中提取准确的定价
- 智能滚动：动态滚动以加载更多产品
- RESTful API：用于自动化任务的简单HTTP端点
- 实时跟踪：使用详细的执行日志监控任务进度
- 异步处理：使用FastAPI执行后台任务
______________________________________________________________________
快速演示
目标: *“去亚马逊找到第五台笔记本电脑的价格”*
结果：代理人自主：
- 导航到Amazon.com
- 在搜索框中填写“笔记本电脑”
- 点击搜索按钮
- 向下滚动3次以加载更多结果
- 摘录第五台笔记本电脑的价格： $379.99
全力以赴 8次迭代 和 约15秒!
______________________________________________________________________
建筑
playwright-browser-automation/
├── agent.py # Core automation engine with error handling
├── main.py # FastAPI server with async task management
├── prompt.py # Amazon-optimized system prompt
├── install.sh # One-click setup script
├── .env # Configuration (Gemini API key)
└── requirements.txt # Python dependencies
关键设计原则
- 网站特定提示：系统提示
prompt.py已优化 仅限Amazon.com。这使得代理在亚马逊产品搜索中非常精确，但需要对其他网站进行及时修改。
- 可配置用于其他站点想让eBay或沃尔玛自动化吗？只需修改
prompt.py使用特定于站点的选择器和模式。
______________________________________________________________________
安装
先决条件
- Python 3.12+
- Node.js 20+（用于Playwright MCP服务器）
- Gemini API密钥(在这里买一个)
设置命令
- 克隆仓库
git clone https://github.com/snehanshu-raj/playwright-browser-automation.git
cd playwright-browser-automation- 运行安装脚本
chmod +x install.sh
./install.sh- 重要提示：将Gemini API密钥添加到.env文件
echo "GEMINI_API_KEY=your_api_key_here" > .env- 启动API服务器
python3 main.py- 就是这样！您的API现在正在运行
http://localhost:8000 - 斯瓦格：
http://localhost:8000/docs
______________________________________________________________________
用法
1.提交自动化任务
curl -X POST http://localhost:8000/automate \
-H "Content-Type: application/json" \
-d '{
"goal": "Go to Amazon and find the price of the first laptop",
"max_iterations": 15
}'答复：
{
"task_id": "abc-123-def-456",
"status": "pending",
"message": "Task submitted successfully. Check status at /task/abc-123-def-456"
}您将看到浏览器自动打开，然后您的查询将逐步执行。
2.检查任务状态
curl http://localhost:8000/task/abc-123-def-456答复：
{
"task_id": "abc-123-def-456",
"status": "completed",
"goal": "Go to Amazon and find the price of the first laptop",
"result": "The price is \$899.00",
"iterations_used": 5,
"started_at": "2025-10-19T09:00:01",
"completed_at": "2025-10-19T09:00:15",
"history": [
"playwright_navigate succeeded",
"playwright_fill succeeded",
"playwright_click succeeded",
"playwright_evaluate returned: \"899.\""
],
"execution_log": [...]
}3.交互式API文档
也可以通过Swagger尝试：服务器运行时访问的URL：
- Swagger 用户界面: http://localhost:8000/docs
______________________________________________________________________
重要：使用 每次部署一个网站 为了获得最大的精度。LLM在专注、针对特定地点的指导下表现最佳。
______________________________________________________________________
API终点
| 方法 | 端点 | 描述 |
|---|---|---|
GET | / | API信息和示例 |
POST | /automate | 提交自动化任务 |
GET | /task/{task_id} | 获取任务状态和结果 |
GET | /tasks | 列出所有任务 |
DELETE | /task/{task_id} | 删除任务 |
GET | /health | 健康检查 |
______________________________________________________________________
项目结构
.
├── agent.py # Automation engine
│ ├── run_agent() # Main execution loop
│ ├── Error handling # Retry logic and failure recovery
│ └── Tool calling # Playwright tool orchestration
│
├── .env # Your Gemini API key goes here
│
├── main.py # FastAPI application
│ ├── Background tasks # Async task processing
│ ├── REST endpoints # API routes
│ └── Task management # Status tracking
│
├── prompt.py # AI System Prompt
│ ├── Tool descriptions # Available Playwright actions
│ ├── Site selectors # Amazon-specific patterns
│ ├── Scrolling rules # Scroll behavior
│ └── Decision logic # When to stop/continue
│
└── install.sh # Setup automation script
完整流程（一目了然）：
┌─────────────────────────────────────────────────────────────────────────┐
│ USER REQUEST │
│ curl -X POST /automate -d '{"goal": "Find laptop price on Amazon"}' │
└────────────────────────────────┬────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────┐
│ FASTAPI SERVER (main.py) │
│ • Receives HTTP request │
│ • Generates unique task_id │
│ • Creates task entry in memory store │
│ • Spawns background task │
│ • Returns 200 OK with task_id immediately │
└────────────────────────────────┬────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────┐
│ BACKGROUND TASK EXECUTOR │
│ • Calls run_agent(goal, max_iterations) │
│ • Initializes log_callback for real-time logging │
│ • Updates task status: pending → running │
└────────────────────────────────┬────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────┐
│ AGENT ENGINE (agent.py) │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ INITIALIZATION │ │
│ │ • Connect to Playwright MCP Server (via npx) │ │
│ │ • Load tools (32 Playwright automation tools) │ │
│ │ • Load SYSTEM_PROMPT_MANUAL from prompt.py │ │
│ └─────────────────────────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ ITERATION LOOP (max 15 iterations be deafult) │ │
│ │ │ │
│ │ FOR i in range(max_iterations): │ │
│ │ 1. Build prompt with: │ │
│ │ - System instructions (Amazon-specific) │ │
│ │ - User goal │ │
│ │ - Previous action history (last 3 steps) │ │
│ │ - Reminder to return FINAL_ANSWER when done │ │
│ │ │ │
│ │ 2. Call Gemini 2.0 Flash Lite API │ │
│ │ → Returns: TOOL_CALL or FINAL_ANSWER │ │
│ │ │ │
│ │ 3. Parse response: │ │
│ │ - Extract tool_name and parameters │ │
│ │ - Split on "|" delimiter (except playwright_evaluate) │ │
│ │ │ │
│ │ 4. Execute tool via MCP: │ │
│ │ session.call_tool(tool_name, args) │ │
│ │ │ │
│ │ 5. Capture result and update history │ │
│ │ - Log to callback for API tracking │ │
│ │ - Detect failures/success │ │
│ │ - Append to action history │ │
│ │ │ │
│ │ 6. Check termination: │ │
│ │ IF "FINAL_ANSWER:" in response: │ │
│ │ → Extract answer and RETURN │ │
│ │ ELSE: │ │
│ │ → Continue to next iteration │ │
│ └─────────────────────────────────────────────────────────────┘ │
└────────────────────────────────┬────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────┐
│ PLAYWRIGHT MCP SERVER (Node.js) │
│ • Running as child process via npx │
│ • Manages Chromium browser instance (headless in Docker via xvfb) │
│ • Provides 32 tools: │
│ - playwright_navigate │
│ - playwright_fill │
│ - playwright_click │
│ - playwright_evaluate (JavaScript execution) │
│ - playwright_screenshot │
│ - etc. │
└────────────────────────────────┬────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────┐
│ CHROMIUM BROWSER │
│ • Launches via Playwright │
│ • Navigates to Amazon.com │
│ • Executes DOM interactions: │
│ - Fill search box: input[id="twotabsearchtextbox"] │
│ - Click search: input[id="nav-search-submit-button"] │
│ - Scroll: window.scrollBy(0, 800) │
│ - Extract price: document.querySelector('.a-price-whole') │
└────────────────────────────────┬────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────┐
│ GEMINI AI (LLM) │
│ Model: gemini-2.0-flash-lite │
│ • Receives prompt with: │
│ - Goal: "Find laptop price on Amazon" │
│ - Available tools │
│ - Action history │
│ • Reasons about next action │
│ • Returns structured response: │
│ TOOL_CALL: playwright_navigate | https://amazon.com │
│ TOOL_CALL: playwright_fill | input[id="..."] | laptop │
│ TOOL_CALL: playwright_evaluate | document.querySelector(...) │
│ FINAL_ANSWER: The price is $899.00 │
└─────────────────────────────────────────────────────────────────────────┘
│
┌────────────┴────────────┐
▼ ▼
┌───────────────────────┐ ┌──────────────────────┐
│ SUCCESS PATH │ │ FAILURE PATH │
│ │ │ │
│ • Extract result │ │ • Retry with alt │
│ • Update task status │ │ approach │
│ • Return to user │ │ • Log failure │
│ │ │ • Continue loop │
└───────────────────────┘ └──────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────┐
│ API RESPONSE │
│ GET /task/{task_id} │
│ { │
│ "task_id": "abc-123", │
│ "status": "completed", │
│ "result": "The price is $899.00", │
│ "iterations_used": 5, │
│ "history": ["Navigate", "Fill", "Click", "Extract"], │
│ "execution_log": [...] │
│ } │
└─────────────────────────────────────────────────────────────────────────┘______________________________________________________________________