Groqcloud MCP Server

🚀 Groq MCP Server

A comprehensive and intelligent Model Context Protocol (MCP) server designed to integrate your application with the powerful Groq API, providing optimized access to the world's fastest AI models.

It acts as a smart bridge, enabling compatible clients (such as Claude Desktop) to utilize various Groq models for text completion, audio transcription, vision analysis, and batch processing.

✨ Features

🧠 Supported Models

This server is configured to manage and route requests to a wide range of Groq models, including:

• Text Completion (LLMs):
  • llama-3.1-8b-instant
  • llama-3.3-70b-versatile
  • deepseek-r1-distill-llama-70b
  • qwen/qwen3-32b (and qwen-qwq-32b for mathematics)
  • compound-beta, compound-beta-mini
  • allam-2-7b, gemma2-9b-it, llama3-70b-8192, llama3-8b-8192
  • mistral-saba-24b
• Security (Prompt/Content Guard):
  • meta-llama/llama-guard-4-12b
  • meta-llama/llama-prompt-guard-2-22m, meta-llama/llama-prompt-guard-2-86m
• Vision (Multimodal):
  • meta-llama/llama-4-maverick-17b-128e-instruct
  • meta-llama/llama-4-scout-17b-16e-instruct
• Audio (Speech-to-Text):
  • whisper-large-v3, whisper-large-v3-turbo
  • distil-whisper-large-v3-en
• Text-to-Speech:
  • playai-tts, playai-tts-arabic

⚡ Advanced Features

• Intelligent Routing (ModelRouter): Dynamically selects the ideal model based on priorities (speed, quality, cost, reasoning, mathematics, multilingual), prompt complexity, and specific capabilities (vision, audio).
• Controlled Rate Limiting: Intelligently manages request and token limits per minute (RPM/TPM) for each model, optimizing API usage.
• Optimized Caching: In-memory cache system with configurable TTL (Time-To-Live) for LLM responses, reducing latency and redundant API calls.
• Detailed Metrics: Collects comprehensive usage, performance (latency, throughput), error, and model distribution metrics for analysis and monitoring.
• Robust Error Handling: Centralized error handling system with automatic retry capabilities for API requests, increasing resilience.
• Batch Processing: Supports Groq's batch processing tool, enabling efficient submission of large volumes of requests at a lower cost.
• Structured Logging and Debugging: Professional logging system with Winston, generating structured logs and directing colored output to stderr during development for easy debugging and monitoring.

📦 Installation

Prerequisites

• Node.js: Version v20.17.0 or higher, or v22.9.0 or higher. It is recommended to use NVM (or nvm-windows) to manage Node.js versions.
• npm: Node.js package manager (usually included with Node.js and compatible with the recommended versions).
• TypeScript: Version 5 or higher.
• Groq API Key: Required to authenticate requests to the Groq API. Obtain yours at https://console.groq.com/keys.

Quick Installation

1. Clone the repository:

   git clone [https://github.com/AyrtonFelipe/GroqCloud-MCP_server.git]
   cd groq-mcp-server
   

2. Install project dependencies:

   npm install
   # Also install the library for converting Zod schemas to JSON Schema
   npm install zod-to-json-schema
   

3. Configure environment variables: Create a .env file at the root of the project (if it doesn't exist, you can copy it from .env.example if provided):

   cp .env.example .env
   

   Edit the .env file with your Groq API key:

   GROQ_API_KEY="your_groq_api_key_here"
   

   (Optional: configure other variables such as LOG_LEVEL as needed.)

4. Update src/config/models.json: This file defines the Groq models that your server will use and expose.

   • Remove entries for models that are no longer available or that you don't want to use (check the latest lists in the Groq console).
   • Add all models from the "Supported Models" list (above) that are not already present. For each new model, you must fill in all its properties (name, description, capabilities, costPer1kTokens, rateLimits, etc.) by consulting the official Groq documentation for accurate values.
   • Adjust the modelSelectionRules and complexityThresholds sections in models.json to reflect the models you have and the desired selection logic (e.g., for reasoning, mathematical, multilingual priorities).

5. Update src/config/constants.ts:

   • Synchronize the RATE_LIMITS constant with the models in your models.json. Ensure that each model in models.json has a corresponding entry in RATE_LIMITS with accurate rpm (requests per minute) and tpm (tokens per minute) values (consult the Groq documentation for the latest values).
   • Also update the z.enum in your tool files (src/tools/*.ts) to include the new models you want to expose to the client.

6. Compile the project:

   npm run build
   

7. Start the server:

   npm start
   

   Your server will be active and waiting for connections via stdin/stdout.

💻 Usage with Claude Desktop

Once your MCP server is running locally, Claude Desktop should be able to discover it and use its tools:

1. Start Claude Desktop.
2. Check Tools: The Groq tools (groq_text_completion, groq_audio_transcription, groq_vision_analysis, groq_batch_processing) should appear enabled in the Claude Desktop interface (usually in the tools or integration menu).
3. Interact: Start chatting with Claude and ask it to use the tools. Examples:
   • Use groq_text_completion to generate a text about Groq's AI capabilities.
   • With the groq_text_completion tool, analyze the financial data { data: [100, 250, 80, 400] } and use the model: llama-3.3-70b-versatile
   • groq_audio_transcription: transcribe the file 'path/to/your/audio.mp3' using 'whisper-large-v3-turbo'.
   • groq_vision_analysis: describe the image at 'https://example.com/your-image.jpg' using 'meta-llama/llama-4-scout-17b-16e-instruct'.

📚 Project Structure

.
├── src/
│   ├── config/
│   │   ├── constants.ts         # System constants (RATE_LIMITS, API_ENDPOINTS, etc.)
│   │   └── models.json          # Detailed definitions of Groq models
│   ├── tools/
│   │   ├── audio-transcription.ts # Audio transcription tool
│   │   ├── batch-processing.ts    # Batch processing tool
│   │   ├── model-router.ts        # Central logic for model selection
│   │   ├── text-completion.ts     # Text completion tool
│   │   └── vision-analysis.ts     # Vision analysis tool
│   │   └── ... (new tools like Text-to-Speech, if implemented)
│   ├── types/
│   │   └── groq-types.ts          # TypeScript type definitions for Groq models
│   ├── utils/
│   │   ├── cache-manager.ts       # Cache manager
│   │   ├── error-handler.ts       # Centralized error handling and retries
│   │   ├── logger.ts              # Logging configuration with Winston
│   │   ├── metrics-tracker.ts     # Collection of usage and performance metrics
│   │   └── rate-limiter.ts        # Rate limiting implementation
│   └── server.ts                  # Main entry point for the MCP server
├── dist/                          # TypeScript compilation output
├── logs/                          # Application logs
├── .env                           # Environment variables (e.g., GROQ_API_KEY)
├── .gitignore                     # Files and folders to be ignored by Git
├── package.json                   # Project metadata and dependencies
├── tsconfig.json                  # TypeScript compiler configurations
└── README.md                      # This file

🔧 Next Steps (Scalability for Web)

This project is configured for local use with StdioServerTransport. To scale to a web server environment (e.g., production), the following considerations would be crucial:

• Communication Transport: Migrate from StdioServerTransport to a web transport such as Server-Sent Events (SSE) or WebSockets for communication with web clients.
• Starting Mechanism: Adapt the server lifecycle to a web framework (e.g., Express, Fastify) that listens on HTTP/HTTPS ports.
• Protocol Implementation: Implement HTTP routes that map to MCP JSON-RPC calls.
• Security: Add authentication (e.g., JWT), authorization, HTTPS, and CORS configuration to protect the server.
• Scalability: Implement strategies to handle multiple clients and high traffic (e.g., load balancing, Node.js clusters, distributed caches like Redis for CacheManager and RateLimiter).

Developed by Ayrton Felipe.