Multi Llm Cross Check MCP Server

🚀 Multi-LLM Cross-Check MCP Server

A Model Control Protocol (MCP) server that allows for the simultaneous cross-verification of responses from multiple large language model providers. This server can be integrated with Claude Desktop as an MCP server, providing a unified interface to query different LLM APIs.

✨ Features

• Query multiple LLM providers in parallel.
• Currently supported providers:
  • OpenAI (ChatGPT)
  • Anthropic (Claude)
  • Perplexity AI
  • Google (Gemini)
• Asynchronous parallel processing to speed up response times.
• Easy integration with Claude Desktop.

📦 Installation

Prerequisites

• Python 3.8 or higher.
• API keys for the LLM providers you plan to use.
• The uv package manager (install with pip install uv).

Steps

1. Clone this repository:

git clone https://github.com/lior-ps/multi-llm-cross-check-mcp-server.git
cd multi-llm-cross-check-mcp-server

2. Initialize the uv environment and install the requirements:

uv venv
uv pip install -r requirements.txt

3. Configure in Claude Desktop: Create a file named claude_desktop_config.json and place it in your Claude Desktop configuration directory. The content should be as follows:

{
    "mcp_servers": [
        {
            "command": "uv",
            "args": [
                "--directory",
                "/multi-llm-cross-check-mcp-server",
                "run",
                "main.py"
            ],
            "env": {
                "OPENAI_API_KEY": "your_openai_key",  // Set to your OpenAI key from https://platform.openai.com/api-keys
                "ANTHROPIC_API_KEY": "your_anthropic_key",  // Set to your Anthropic key from https://console.anthropic.com/account/keys
                "PERPLEXITY_API_KEY": "your_perplexity_key",  // Set to your Perplexity key from https://www.perplexity.ai/settings/api
                "GEMINI_API_KEY": "your_gemini_key"  // Set to your Gemini key from https://makersuite.google.com/app/apikey
            }
        }
    ]
}

⚠️ Important Note

• Only add the API keys for the LLM providers you plan to use. The server will automatically skip providers without a configured API key.
• You may need to put the full path of the uv executable in the command field. You can get this path by running which uv (MacOS/Linux) or where uv (Windows).

💻 Usage Examples

Basic Usage

After configuration:

1. The service will start automatically when you open Claude Desktop.
2. You can use the "cross_check" tool in the conversation and request "cross-verification with other LLMs".
3. Provide a prompt, and it will return responses from all configured LLM providers.

API Response Format

The server returns a dictionary containing responses from each LLM provider:

{
    "ChatGPT": { ... },
    "Claude": { ... },
    "Perplexity": { ... },
    "Gemini": { ... }
}

If there is an error with a certain model (e.g., invalid key or network issue), that part will be marked as an error, but it will not affect the responses of other models.

📚 Documentation

Expansion and Improvement

• Support for more LLM providers: such as Amazon AI, Microsoft Azure, Hugging Face, etc.
• Enhanced error handling: add more detailed logging and error information reporting.
• Result analysis functionality: add statistical data or visualization tools to help users compare the responses of different models.
• High availability design: implement load balancing and failover mechanisms to ensure that other providers can still be used when one provider is unavailable.

The core idea of this project is to unify multiple LLM services through the MCP protocol, improve performance using parallel requests, and provide flexible configuration to meet the needs of different users.