Logstash MCP Server

🚀 Logstash MCP Server

A Model Context Protocol (MCP) server for interacting with Logstash instances, offering comprehensive monitoring and management tools.

⚠️ Important Note

This repository is vibe coded, AI generated and not tested properly. Use it with your own risk.

🚀 Quick Start

After starting the server with python3 logstash_mcp_server.py, you can use the following JSON - RPC commands:

1. Initialize (Required First)

{"jsonrpc": "2.0", "id": 0, "method": "initialize", "params": {"protocolVersion": "2024-11-05", "capabilities": {}, "clientInfo": {"name": "test-client", "version": "1.0.0"}}}

2. Check Connectivity

{"jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": {"name": "logstash_check_connectivity", "arguments": {}}}

3. Health Check

{"jsonrpc": "2.0", "id": 2, "method": "tools/call", "params": {"name": "logstash_health_check", "arguments": {}}}

4. List All Tools

{"jsonrpc": "2.0", "id": 3, "method": "tools/list"}

5. Get Node Info

{"jsonrpc": "2.0", "id": 4, "method": "tools/call", "params": {"name": "logstash_node_info", "arguments": {}}}

✨ Features

Monitoring Tools

• Node Information: Get Logstash version, build info, and settings.
• Node Statistics: JVM, process, and pipeline metrics.
• Pipeline Statistics: Monitor individual or all pipeline performance.
• Hot Threads: Debug performance issues with thread analysis.
• Health Check: Comprehensive health assessment with recommendations.
• Connectivity Check: Verify connection to Logstash with detailed diagnostics.

Management Tools

• Pipeline Reload: Reload specific pipeline configurations.
• Plugin Listing: View all installed Logstash plugins.
• JVM Statistics: Detailed memory and garbage collection metrics.
• Grok Patterns: List available Grok patterns for log parsing.

📦 Installation

1. Install dependencies:

pip install -r requirements.txt

2. Set up environment variables (optional):

export LOGSTASH_API_BASE="http://your-logstash-host:9600"

💻 Usage Examples

Basic Usage

# The MCP server will automatically analyze:
# - JVM memory usage
# - Pipeline performance
# - Queue statistics
# - Provide recommendations for optimization

Advanced Usage

# Monitor specific pipeline performance
# Get detailed statistics for troubleshooting
# Track event processing rates and errors

# Use hot threads analysis to identify bottlenecks
# Monitor JVM statistics for memory issues
# Track pipeline queue usage

📚 Documentation

Web UI

The project includes a web - based user interface for easy interaction with your Logstash instance.

Running the Web UI

1. Start the web interface:

python3 web_ui.py

2. Open your browser and navigate to:

http://localhost:5001

Web UI Features

The web interface provides:

• Interactive Dashboard: Visual interface to access all Logstash monitoring tools.
• Real - time Monitoring: Check connectivity, node stats, and pipeline performance.
• Health Analysis: Comprehensive health checks with visual feedback.
• Pipeline Management: View statistics for individual or all pipelines.
• Performance Debugging: Hot threads analysis and JVM statistics.
• Plugin Management: Browse installed Logstash plugins.

Web UI Configuration

The web UI uses the same configuration as the MCP server:

• Default Logstash URL: http://localhost:9600.
• Override with: LOGSTASH_API_BASE environment variable.
• Web interface runs on: http://localhost:5001.

Example with custom Logstash URL:

export LOGSTASH_API_BASE="http://your-logstash-host:9600"
python3 web_ui.py

Available Tools (12 Total)

Health Check Analysis

The health check tool analyzes:

• Connectivity Verification: Tests connection to Logstash before other checks.
• JVM Memory Usage: Warns if heap usage exceeds 80%.
• Pipeline Performance: Detects pipelines with filtered but no output events.
• Queue Usage: Identifies large queue sizes that may impact performance.

Integration with ELK Stack

This MCP server is designed to work alongside Elasticsearch diagnostics and can help:

• Monitor Logstash performance feeding into your Elasticsearch cluster.
• Identify pipeline bottlenecks that may contribute to indexing delays.
• Optimize Logstash configuration for better cluster performance.

Based on your Elasticsearch cluster analysis showing high shard counts, ensure your Logstash pipelines are optimized for efficient indexing patterns.

Error Handling

The server includes comprehensive error handling for:

• Connection failures to Logstash API.
• Invalid pipeline IDs.
• API response errors.
• Network timeouts.
• Detailed error messages with troubleshooting suggestions.

Testing

Run the test suite to verify everything works:

python3 test_mcp_server.py

The test suite includes:

• Server initialization tests.
• Tool listing verification.
• Mocked health check tests.
• Error handling validation.

Security Considerations

• The server connects to Logstash API endpoints.
• Ensure proper network security between MCP server and Logstash.
• Consider authentication if your Logstash instance requires it.
• Monitor API access logs for security auditing.