Lior Ps Multi Llm Cross Check MCP Server

🚀 Multi-LLM Cross-Check MCP Server

A Model Control Protocol (MCP) server that allows cross-checking responses from multiple Large Language Model (LLM) providers simultaneously. This server integrates with the desktop version of Claude and serves as an MCP server, offering a unified interface to query different LLM APIs.

✨ Features

• Query multiple LLM providers in parallel.
• Currently supported providers:
  • OpenAI (ChatGPT)
  • Anthropic (Claude)
  • Perplexity AI
  • Google (Gemini)
• Asynchronous parallel processing to speed up responses.
• Easy integration with the desktop version of Claude.

📦 Installation

Prerequisites

• Python 3.8 or higher.
• API keys for the selected LLM providers.
• uv package manager (install using pip install uv).

Steps

1. Clone this repository:

git clone https://github.com/lior-ps/multi-llm-cross-check-mcp-server.git
cd multi-llm-cross-check-mcp-server

2. Initialize the uv environment and install the requirements:

uv venv
uv pip install -r requirements.txt

3. Configure in the desktop version of Claude: Create a file named claude_desktop_config.json in your desktop Claude configuration directory with the following content:

{
    "mcp_servers": [
        {
            "command": "uv",
            "args": [
                "--directory",
                "/multi-llm-cross-check-mcp-server",
                "run",
                "main.py"
            ],
            "env": {
                "OPENAI_API_KEY": "your_openai_key",  // From https://platform.openai.com/api-keys
                "ANTHROPIC_API_KEY": "your_anthropic_key",  // From https://console.anthropic.com/account/keys
                "PERPLEXITY_API_KEY": "your_perplexity_key",  // From https://www.perplexity.ai/settings/api
                "GEMINI_API_KEY": "your_gemini_key"  // From https://makersuite.google.com/app/apikey
            }
        }
    ]
}

⚠️ Important Note

1. You only need to add the API keys for the LLM providers you want to use. The server will skip providers without a configured API key.
2. You may need to place the full path of the uv executable in the command field. You can obtain it by running which uv on MacOS/Linux or where uv on Windows.

💻 Usage Examples

Basic Usage

After completing the configuration:

1. The server will start automatically when you open the desktop version of Claude.
2. You can use the cross_check tool by requesting "Cross-check with other LLMs".
3. Provide a prompt, and it will return responses from all configured LLM providers.

API Response Format

The server returns a dictionary of responses from each LLM provider:

{
    "ChatGPT": { ... },
    "Claude": { ... },
    "Perplexity": { ... },
    "Gemini": { ... }
}

Error Handling

• If the API key for a specific LLM is not provided, the provider will be skipped.
• API errors are captured and returned in the response.
• The responses from each LLM are independent, so an error from one provider does not affect others.

🤝 Contribution

Contributions are welcome! Please feel free to submit pull requests.

📄 License

This project is released under the MIT License - see the LICENSE file for details.