MCP Server Python

🚀 MCP Server: Documentation Retrieval & Web Scraping (uv + FastMCP)

This project offers a minimal, asynchronous MCP (Model Context Protocol) server. It provides a tool for retrieving and cleaning official documentation content of popular libraries in the AI / Python ecosystem. Here's what it uses:

• fastmcp to define and run the MCP server over stdio.
• httpx for asynchronous HTTP calls.
• serper.dev for Google - like search (via API).
• groq API (LLM) to transform raw HTML into readable text chunks.
• python - dotenv for environment variable management.
• uv as the package manager & runner (fast, lockfile - based, Python 3.11+).

✨ Features

• The search is restricted to official documentation domains (uv, langchain, openai, llama - index).
• The tool get_docs(query, library) returns concatenated and cleaned sections with SOURCE: labels.
• It has a streaming - safe asynchronous design (chunking large HTML pages before LLM cleaning).
• There is a separate client.py that demonstrates how to connect as an MCP client, call the tool, and then post - process with an LLM.

🚀 Quick Start

Prerequisites

• Python 3.11+
• uv installed (https://docs.astral.sh/uv/)
• API keys for: SERPER_API_KEY, GROQ_API_KEY

1. Clone & Install

git clone <your - repo - url> mcp - server - python
cd mcp - server - python
uv sync

This will create or refresh a .venv based on pyproject.toml + uv.lock.

2. Environment Variables

Create a .env file in the project root:

SERPER_API_KEY=your_serper_api_key_here
GROQ_API_KEY=your_groq_api_key_here

Optionally, add other model settings if you plan to extend the functionality later.

3. Run the MCP Server Directly

uv run mcp_server.py

The server will start and wait on stdio (no extra output unless you add logging). It registers the tool get_docs.

4. Use the Provided Client

uv run client.py

You should see something like:

Available tools: ['get_docs']
ANSWER: <model - produced answer referencing SOURCE lines>

If the list is empty, make sure the server started correctly and no exceptions were raised (add logging as described below).

💻 Usage Examples

Basic Usage

from mcp.client.stdio import stdio_client
from mcp import ClientSession, StdioServerParameters
import asyncio

async def demo():
    params = StdioServerParameters(command="uv", args=["run", "mcp_server.py"])
    async with stdio_client(params) as (r, w):
        async with ClientSession(r, w) as session:
            await session.initialize()
            tools = await session.list_tools()
            print([t.name for t in tools.tools])
            docs = await session.call_tool("get_docs", {"query": "install", "library": "uv"})
            print(docs.content[:500])

asyncio.run(demo())

📚 Documentation

Tool: get_docs

Signature:

get_docs(query: str, library: str) -> str

Supported libraries (keys): uv, langchain, openai, llama - index.

Flow:

1. Build a site - restricted query: site:<docs - domain> <query>.
2. Call the Serper API for organic results.
3. Fetch each result URL (asynchronously) via httpx.
4. Split HTML into ~4000 - char chunks (for memory safety and to respect LLM limits).
5. Clean each chunk using the Groq LLM (openai/gpt - oss - 20b) with a system prompt.
6. Concatenate and label each block with SOURCE: <url> for traceability.

The returned value is a large text blob suitable for retrieval - augmented prompting, preserving source attribution lines.

Architecture

Dependency Notes

Core runtime dependencies (from pyproject.toml):

• fastmcp – MCP server helper.
• httpx – Async HTTP client.
• groq – Groq API client.
• python - dotenv – Load variables from .env.
• trafilatura – Heuristic content extraction (currently partially used / can be extended).

💡 Usage Tip

If you add more scraping tools, reuse a single httpx.AsyncClient for better performance.

Logging & Debugging

To see what the server is doing, you can temporarily add:

import logging, sys
logging.basicConfig(level=logging.INFO, stream=sys.stderr)

Place it near the top of mcp_server.py after imports. Since the protocol uses stdout for JSON - RPC, send logs to stderr only.

Common issues:

• Empty tool list: The server exited early or crashed—add logging.
• SERPER_API_KEY missing → 401 or empty search results.
• GROQ_API_KEY missing → LLM cleaning fails (exception in get_response_from_llm).
• Network timeouts: Adjust timeout in httpx.AsyncClient calls.

Extending

Here are some ideas for extending the project:

• Add a caching layer (e.g., sqlite or an in - memory dict) to avoid re - fetching the same URLs.
• Parallelize URL fetching and cleaning with asyncio.gather() (be mindful of rate limits and LLM costs).
• Add another tool (e.g., summarize_diff, list_endpoints).
• Provide structured JSON output (list of sources + cleaned text) instead of a concatenated string.
• Add tests using pytest + pytest - asyncio (mock Serper + LLM APIs).

Running With Active Virtualenv

If you have an already activated virtual environment and want to use it instead of the project’s pinned environment, you can force uv to target it:

uv run --active client.py

Otherwise, uv will warn that your active $VIRTUAL_ENV differs from the project .venv but continue using the project environment.

Troubleshooting Cheat Sheet

📄 License

Add a license section here (e.g., MIT) if you intend to distribute.

🔧 Contributing

1. Fork the repository and create a new branch.
2. Run uv sync.
3. Add tests for new tools (if added).
4. Open a pull request with a clear description.

🚧 Roadmap (Optional)

• [ ] Add JSON schema metadata for tool params.
• [ ] Implement a structured response format (list of {source, text}).
• [ ] Add a caching layer.
• [ ] Add rate limiting/backoff.
• [ ] Add a CI workflow (lint + tests).

🙏 Acknowledgments

• Serper.dev for the search API
• Groq for fast OSS model serving
• Astral for uv
• MCP ecosystem for the protocol foundation