Pavanjava Kafka MCP Server

🚀 Kafka MCP Server

A Message Context Protocol (MCP) server integrated with Apache Kafka, providing message publishing and consumption capabilities for Large Language Models (LLMs) and agent applications.

🚀 Quick Start

The Kafka MCP Server is a project that enables AI models to interact with Kafka topics through a standardized interface. It supports the following operations:

• Publishing messages to Kafka topics.
• Consuming messages from Kafka topics.

✨ Features

• Allows AI models to interact with Kafka topics via a standardized interface.
• Supports message publishing and consumption operations.

📦 Installation

Prerequisites

• Python 3.8+
• An Apache Kafka instance
• Python dependencies (see the installation section)

Steps

1. Clone the repository:

   git clone <repository-url>
   cd <repository-directory>
   

2. Create and activate a virtual environment:

   python -m venv venv
   source venv/bin/activate  # On Windows, use: venv\Scripts\activate
   

3. Install the required dependencies:

   pip install -r requirements.txt
   

   If the requirements.txt file is not available, install the following packages:

   pip install aiokafka python-dotenv pydantic-settings mcp-server
   

📚 Documentation

Configuration

Create a .env file in the project root directory and add the following variables:

# Kafka configuration
KAFKA_BOOTSTRAP_SERVERS=localhost:9092
TOPIC_NAME=your-topic-name
IS_TOPIC_READ_FROM_BEGINNING=False
DEFAULT_GROUP_ID_FOR_CONSUMER=kafka-mcp-group

# Optional: Custom tool descriptions
# TOOL_PUBLISH_DESCRIPTION="Custom description for the publishing tool"
# TOOL_CONSUME_DESCRIPTION="Custom description for the consumption tool"

Usage

Running the Server

You can run the server using the provided main.py script:

python main.py --transport stdio

Available transport options:

• stdio: Standard input/output (default)
• sse: Server-Sent Events

Integration with Claude Desktop

To use this Kafka MCP server with Claude Desktop, add the following configuration to your Claude Desktop configuration file:

{
    "mcpServers": {
        "kafka": {
            "command": "python",
            "args": [
                "<PATH TO PROJECTS>/main.py"
            ]
        }
    }
}

Replace <PATH TO PROJECTS> with the absolute path of the project.

Project Structure

• main.py: The entry point of the application.
• kafka.py: The implementation of the Kafka connector.
• server.py: The implementation of the MCP server with Kafka interaction tools.
• settings.py: Manages the configuration using Pydantic.

Available Tools

kafka-publish

Publishes messages to the configured Kafka topic.

kafka-consume

Consumes messages from the configured Kafka topic.

⚠️ Important Note

Once a message is read, it cannot be read again using the same group ID.