Voice Agent With SQL And Web Fallback

MCP Powered Voice Agent

A fully local, intelligent voice agent built using the Model Context Protocol (MCP). It listens to your voice, transcribes it offline using Whisper, understands your intent via a local LLM (Ollama), and fetches answers from a structured SQL database or falls back to a DuckDuckGo web search.

It comes with both a sleek PyQt5 Desktop GUI and a Command Line Interface.

Features

• 🎙️ 100% Local Voice Transcription: Uses OpenAI's Whisper model (base) to convert speech into text privately and offline.
• 🧠 Local LLM Orchestration: Powered by llama3.2 running via Ollama and orchestrated with LangChain's Tool Calling Agent.
• 🗄️ Database MCP Tool: A built-in SQLite database tool that the agent can query for structured information (e.g., querying staff schedules).
• 🌐 Web Search Fallback: If the local database doesn't have the answer, the agent smartly falls back to DuckDuckGo to answer general knowledge questions.
• 🖥️ Desktop GUI: A responsive PyQt5 interface with a "Push to Talk" chat window.

Prerequisites

1. Python 3.12 is highly recommended (Python 3.14 is currently incompatible with some LangChain/Pydantic binary wheels).
2. Ollama installed and running with the llama3.2 model.

    ollama run llama3.2

1. FFmpeg installed on your system (required by Whisper for audio processing).

* Mac: brew install ffmpeg * Ubuntu/Debian: sudo apt update && sudo apt install ffmpeg

1. PortAudio installed on your system (required by PyAudio).

* Mac: brew install portaudio * Ubuntu/Debian: sudo apt install portaudio19-dev

Installation

1. Clone the repository:

   git clone 
   cd mcp-voice-agent

1. Create and activate a virtual environment:

   python3.12 -m venv .venv
   source .venv/bin/activate  # On Windows: .venv\\Scripts\\activate

1. Install the required dependencies:

   pip install -U pip
   pip install -r requirements.txt

Usage

Ensure your virtual environment is activated and Ollama is running.

Run the Desktop GUI (Default):

python main.py

Click the "Push to Talk" button, ask your question, and wait for the agent to think and respond!

Run the Terminal / CLI Version:

python main.py --cli

Testing the Tools

• Database Tool Trigger: Ask something like *"What is Alice Smith's schedule?"* -> The agent will recognize it needs to check the staff database.
• Web Fallback Trigger: Ask something like *"Who won the Super Bowl in 2024?"* -> The agent will realize the database doesn't know this and will fall back to DuckDuckGo.

Architecture & MCP Details

The Model Context Protocol (MCP) integrates the tools. Although currently bound directly into the LangChain executor for speed and simplicity, the tools in the tools/ directory are built using FastMCP.

This means you can easily detach them and run python tools/database.py to start a standalone MCP server that any standard MCP Client (like Claude Desktop) can connect to!