MCP Ai Hub

🚀 MCP AI Hub

MCP AI Hub is a Model Context Protocol (MCP) server that uses LiteLM to offer unified access to various AI providers. You can chat with OpenAI, Anthropic, and over 100 other AI models through a single, consistent interface.

🚀 Quick Start

1. Install

Choose your preferred installation method:

# Option A: Install from PyPI
pip install mcp-ai-hub

# Option B: Install with uv (recommended)
uv tool install mcp-ai-hub

# Option C: Install from source
pip install git+https://github.com/your-username/mcp-ai-hub.git

Installation Notes:

• uv is a fast Python package installer and resolver.
• The package requires Python 3.10 or higher.
• All dependencies are automatically resolved and installed.

2. Configure

Create a configuration file at ~/.ai_hub.yaml with your API keys and model configurations:

model_list:
  - model_name: gpt-4  # Friendly name you'll use in MCP tools
    litellm_params:
      model: openai/gpt-4  # LiteLM provider/model identifier
      api_key: "sk-your-openai-api-key-here"  # Your actual OpenAI API key
      max_tokens: 2048  # Maximum response tokens
      temperature: 0.7  # Response creativity (0.0-1.0)

  - model_name: claude-sonnet
    litellm_params:
      model: anthropic/claude-3-5-sonnet-20241022
      api_key: "sk-ant-your-anthropic-api-key-here"
      max_tokens: 4096
      temperature: 0.7

Configuration Guidelines:

• API Keys: Replace placeholder keys with your actual API keys.
• Model Names: Use descriptive names you'll remember (e.g., gpt-4, claude-sonnet).
• LiteLM Models: Use LiteLM's provider/model format (e.g., openai/gpt-4, anthropic/claude-3-5-sonnet-20241022).
• Parameters: Configure max_tokens, temperature, and other LiteLM-supported parameters.
• Security: Keep your config file secure with appropriate file permissions (chmod 600).

3. Connect to Claude Desktop

Configure Claude Desktop to use MCP AI Hub by editing your configuration file: macOS: ~/Library/Application Support/Claude/claude_desktop_config.json Windows: %APPDATA%\Claude\claude_desktop_config.json

{
  "mcpServers": {
    "ai-hub": {
      "command": "mcp-ai-hub"
    }
  }
}

4. Connect to Claude Code

claude mcp add -s user ai-hub mcp-ai-hub

✨ Features

• Unified Interface: Single API for all AI providers.
• 100+ Providers: OpenAI, Anthropic, Google, Azure, AWS Bedrock, and more.
• MCP Protocol: Native integration with Claude Desktop and Claude Code.
• Flexible Configuration: YAML-based configuration with Pydantic validation.
• Multiple Transports: stdio, SSE, and HTTP transport options.
• Custom Endpoints: Support for proxy servers and local deployments.

💻 Usage Examples

Basic Usage

Once MCP AI Hub is connected to your MCP client, you can interact with AI models using these tools:

# Primary Chat Tool
chat(model_name: str, message: str | list[dict]) -> str

• model_name: Name of the configured model (e.g., "gpt-4", "claude-sonnet").
• message: String message or OpenAI-style message list.
• Returns: AI model response as string.

# Model Discovery Tools
list_models() -> list[str]

• Returns: List of all configured model names.

get_model_info(model_name: str) -> dict

• model_name: Name of the configured model.
• Returns: Model configuration details including provider, parameters, etc.

Advanced Usage

MCP AI Hub supports multiple transport mechanisms for different use cases:

# Default stdio transport (for MCP clients like Claude Desktop)
mcp-ai-hub

# Server-Sent Events transport (for web applications)
mcp-ai-hub --transport sse --host 0.0.0.0 --port 3001

# Streamable HTTP transport (for direct API calls)
mcp-ai-hub --transport http --port 8080

# Custom config file and debug logging
mcp-ai-hub --config /path/to/config.yaml --log-level DEBUG

Transport Type Details:

CLI Arguments:

• --transport {stdio,sse,http}: Transport protocol (default: stdio).
• --host HOST: Host address for SSE/HTTP (default: localhost).
• --port PORT: Port number for SSE/HTTP (default: 3001; override if you need a different port).
• --config CONFIG: Custom config file path (default: ~/.ai_hub.yaml).
• --log-level {DEBUG,INFO,WARNING,ERROR}: Logging verbosity (default: INFO).

📚 Documentation

Configuration

MCP AI Hub supports 100+ AI providers through LiteLM. Configure your models in ~/.ai_hub.yaml with API keys and custom parameters.

System Prompts

You can define system prompts at two levels:

• global_system_prompt: Applied to all models by default.
• Per-model system_prompt: Overrides the global prompt for that model.

Precedence: model-specific prompt > global prompt. If a model's system_prompt is set to an empty string, it disables the global prompt for that model.

global_system_prompt: "You are a helpful AI assistant. Be concise."

model_list:
  - model_name: gpt-4
    system_prompt: "You are a precise coding assistant."
    litellm_params:
      model: openai/gpt-4
      api_key: "sk-your-openai-api-key"

  - model_name: claude-sonnet
    # Empty string disables the global prompt for this model
    system_prompt: ""
    litellm_params:
      model: anthropic/claude-3-5-sonnet-20241022
      api_key: "sk-ant-your-anthropic-api-key"

Notes:

• The server prepends the configured system prompt to the message list it sends to providers.
• If you pass an explicit message list that already contains a system message, both system messages will be included in order (configured prompt first).

Supported Providers

Major AI Providers:

• OpenAI: GPT-4, GPT-3.5-turbo, GPT-4-turbo, etc.
• Anthropic: Claude 3.5 Sonnet, Claude 3 Haiku, Claude 3 Opus.
• Google: Gemini Pro, Gemini Pro Vision, Gemini Ultra.
• Azure OpenAI: Azure-hosted OpenAI models.
• AWS Bedrock: Claude, Llama, Jurassic, and more.
• Together AI: Llama, Mistral, Falcon, and open-source models.
• Hugging Face: Various open-source models.
• Local Models: Ollama, LM Studio, and other local deployments.

Configuration Parameters:

• api_key: Your provider API key (required).
• max_tokens: Maximum response tokens (optional).
• temperature: Response creativity 0.0-1.0 (optional).
• api_base: Custom endpoint URL (for proxies/local servers).
• Additional: All LiteLM-supported parameters.

Configuration Examples

Basic Configuration:

global_system_prompt: "You are a helpful AI assistant. Be concise."

model_list:
  - model_name: gpt-4
    system_prompt: "You are a precise coding assistant."  # overrides global
    litellm_params:
      model: openai/gpt-4
      api_key: "sk-your-actual-openai-api-key"
      max_tokens: 2048
      temperature: 0.7

  - model_name: claude-sonnet
    litellm_params:
      model: anthropic/claude-3-5-sonnet-20241022
      api_key: "sk-ant-your-actual-anthropic-api-key"
      max_tokens: 4096
      temperature: 0.7

Custom Parameters:

model_list:
  - model_name: gpt-4-creative
    litellm_params:
      model: openai/gpt-4
      api_key: "sk-your-openai-key"
      max_tokens: 4096
      temperature: 0.9  # Higher creativity
      top_p: 0.95
      frequency_penalty: 0.1
      presence_penalty: 0.1

  - model_name: claude-analytical
    litellm_params:
      model: anthropic/claude-3-5-sonnet-20241022
      api_key: "sk-ant-your-anthropic-key"
      max_tokens: 8192
      temperature: 0.3  # Lower creativity for analytical tasks
      stop_sequences: ["\n\n", "Human:"]

Local LLM Server Configuration:

model_list:
  - model_name: local-llama
    litellm_params:
      model: openai/llama-2-7b-chat
      api_key: "dummy-key"  # Local servers often accept any API key
      api_base: "http://localhost:8080/v1"  # Local OpenAI-compatible server
      max_tokens: 2048
      temperature: 0.7

For more providers, please refer to the LiteLLM docs: https://docs.litellm.ai/docs/providers.

🔧 Technical Details

Development

Setup:

# Install all dependencies including dev dependencies
uv sync

# Install package in development mode
uv pip install -e ".[dev]"

# Add new runtime dependencies
uv add package_name

# Add new development dependencies
uv add --dev package_name

# Update dependencies
uv sync --upgrade

Running and Testing:

# Run the MCP server
uv run mcp-ai-hub

# Run with custom configuration
uv run mcp-ai-hub --config ./custom_config.yaml --log-level DEBUG

# Run with different transport
uv run mcp-ai-hub --transport sse --port 3001

# Run tests (when test suite is added)
uv run pytest

# Run tests with coverage
uv run pytest --cov=src/mcp_ai_hub --cov-report=html

Code Quality:

# Format code with ruff
uv run ruff format .

# Lint code
uv run ruff check .

# Type checking with mypy
uv run mypy src/

# Run all quality checks
uv run ruff format . && uv run ruff check . && uv run mypy src/

Troubleshooting

Configuration Issues

Configuration File Problems:

• File Location: Ensure ~/.ai_hub.yaml exists in your home directory.
• YAML Validity: Validate YAML syntax using online validators or python -c "import yaml; yaml.safe_load(open('~/.ai_hub.yaml'))".
• File Permissions: Set secure permissions with chmod 600 ~/.ai_hub.yaml.
• Path Resolution: Use absolute paths in custom config locations.

Configuration Validation:

• Required Fields: Each model must have model_name and litellm_params.
• API Keys: Verify API keys are properly quoted and not expired.
• Model Formats: Use LiteLM-compatible model identifiers (e.g., openai/gpt-4, anthropic/claude-3-5-sonnet-20241022).

API and Authentication Errors

Authentication Issues:

• Invalid API Keys: Check for typos, extra spaces, or expired keys.
• Insufficient Permissions: Verify API keys have necessary model access permissions.
• Rate Limiting: Monitor API usage and implement retry logic if needed.
• Regional Restrictions: Some models may not be available in all regions.

API-Specific Troubleshooting:

• OpenAI: Check organization settings and model availability.
• Anthropic: Verify Claude model access and usage limits.
• Azure OpenAI: Ensure proper resource deployment and endpoint configuration.
• Google Gemini: Check project setup and API enablement.

MCP Connection Issues

Server Startup Problems:

• Port Conflicts: Use different ports for SSE/HTTP transports if defaults are in use.
• Permission Errors: Ensure executable permissions for mcp-ai-hub command.
• Python Path: Verify Python environment and package installation.

Client Configuration Issues:

• Command Path: Ensure mcp-ai-hub is in PATH or use full absolute path.
• Working Directory: Some MCP clients require specific working directory settings.
• Transport Mismatch: Use stdio transport for Claude Desktop/Code.

Performance and Reliability

Response Time Issues:

• Network Latency: Use geographically closer API endpoints when possible.
• Model Selection: Some models are faster than others (e.g., GPT-3.5 vs GPT-4).
• Token Limits: Large max_tokens values can increase response time.

Reliability Improvements:

• Retry Logic: Implement exponential backoff for transient failures.
• Timeout Configuration: Set appropriate timeouts for your use case.
• Health Checks: Monitor server status and restart if needed.
• Load Balancing: Use multiple model configurations for redundancy.

📄 License

MIT License - see LICENSE file for details.

Contributing

We welcome contributions! Please follow these guidelines:

Development Workflow

1. Fork and Clone: Fork the repository and clone your fork.
2. Create Branch: Create a feature branch (git checkout -b feature/amazing-feature).
3. Development Setup: Install dependencies with uv sync.
4. Make Changes: Implement your feature or fix.
5. Testing: Add tests and ensure all tests pass.
6. Code Quality: Run formatting, linting, and type checking.
7. Documentation: Update documentation if needed.
8. Submit PR: Create a pull request with detailed description.

Code Standards

Python Style:

• Follow PEP 8 style guidelines.
• Use type hints for all functions.
• Add docstrings for public functions and classes.
• Keep functions focused and small.

Testing Requirements:

• Write tests for new functionality.
• Ensure existing tests continue to pass.
• Aim for good test coverage.
• Test edge cases and error conditions.

Documentation:

• Update README.md for user-facing changes.
• Add inline comments for complex logic.
• Update configuration examples if needed.
• Document breaking changes clearly.

Quality Checks

Before submitting a PR, ensure:

# All tests pass
uv run pytest

# Code formatting
uv run ruff format .

# Linting passes
uv run ruff check .

# Type checking passes
uv run mypy src/

# Documentation is up to date
# Configuration examples are valid

Issues and Feature Requests

• Use GitHub Issues for bug reports and feature requests.
• Provide detailed reproduction steps for bugs.
• Include configuration examples when relevant.
• Check existing issues before creating new ones.
• Label issues appropriately.