Multi LLM API Gateway

🚀 MCP Server

This project implements a standard context for interaction between AI models and development environments based on FastAPI and MCP (Model Context Protocol). By simplifying model deployment, providing efficient API endpoints, and ensuring consistency in model input and output, it aims to enhance the scalability and maintainability of the system.

🚀 Quick Start

Installation

1. Clone the repository:

   git clone https://github.com/your-repository.git
   cd mcp_server
   

2. Install dependencies:

   pip install -r requirements.txt
   

Usage

Start the server

python mcp_server.py

Run the client

python client.py

✨ Features

• JSON-RPC 2.0 Support: Supports the JSON-based remote procedure call protocol.
• SSE Connection: Supports Server-Sent Events for real-time data transmission.
• Modular Design: The system architecture is modular for easy expansion and maintenance.
• Asynchronous Processing: Adopts an asynchronous programming model to improve performance.

📦 Project Structure

MCP Server
├── mcp_server.py
├── requirements.txt
└── LICENSE

💻 Usage Examples

Basic Usage

API Endpoints

• Root Path: /

  • Method: GET
  • Function: Returns a welcome message.

• API Endpoint: /api

  • Method: POST
  • Function: Handles JSON-RPC requests.

• SSE Endpoint: /stream

  • Method: GET
  • Function: Provides real-time stream data.

Advanced Usage

MCP Protocol Implementation

Initialization Process

1. Client sends an initialization request:

   {
     "id": 1,
     "method": "initialize",
     "params": {
       "model_id": "gpt-4"
     }
   }
   

2. Server response:

   {
     "id": 1,
     "result": true
   }
   

Sampling Request

1. Client sends a sampling request:

   {
     "id": 2,
     "method": "sample",
     "params": {
       "prompt": "Explain the MCP protocol"
     }
   }
   

2. Server response:

   {
     "id": 2,
     "result": {
       "content": "The MCP protocol is a standard for regulating the interaction between AI models and development environments...",
       "usage": {
         "prompt_tokens": 10,
         "completion_tokens": 20,
         "total_tokens": 30
       }
     }
   }
   

Close the Session

{
  "id": 3,
  "method": "shutdown"
}

🔧 Technical Details

Extension Development

Add New Methods

Add new methods in mcp_server.py:

def new_method(self, params):
    # Implement new functionality
    return {"result": "Operation successful"}

Integrate AI Models

Modify the handle_sample method to integrate actual models:

async def handle_sample(self, params: dict) -> dict:
    prompt = params.get("prompt", "")
    response = await openai.ChatCompletion.acreate(
        model="gpt-4",
        messages=[{"role": "user", "content": prompt}]
    )
    content = response.choices[0].message.content
    usage = response.usage
    return {
        "content": content,
        "usage": {
            "prompt_tokens": usage.prompt_tokens,
            "completion_tokens": usage.completion_tokens,
            "total_tokens": usage.total_tokens
        }
    }

Troubleshooting

Common Issues

1. Connection Error: Ensure the server is started and the client uses the correct server URL.
2. 405 Method Not Allowed: Confirm that the client request is sent to the correct API endpoint.
3. SSE Connection Failure: Check network connections and firewall settings.

Logging

View logs for more information:

export PYTHONPATH=.
python -m logging -v DEBUG -m mcp_server

📚 Documentation

• MCP Protocol Specification
• FastAPI Documentation
• JSON-RPC 2.0 Specification
• SSE Specification

📄 License

This project is licensed under the MIT License. For more details, please refer to the LICENSE file.

Chinese Documentation