Llamator MCP Server

🚀 LLAMATOR MCP Server

A production-oriented service wrapper for automated LLM red teaming with LLAMATOR

🚀 Quick Start

This repository offers a production - oriented service wrapper around LLAMATOR for automated LLM red teaming. It provides two integration surfaces:

• HTTP API (FastAPI): Used for job submission, job state retrieval, and artifacts access.
• MCP server (Streamable HTTP transport): For agent/tooling integrations, allowing LLAMATOR runs to be invoked as tools.

Execution is asynchronous and orchestrated via ARQ + Redis. Artifacts are uploaded to MinIO and retrieved through presigned URLs (returned as JSON; the API does not redirect).

✨ Features

• Asynchronous test runs: With durable state persisted in Redis.
• Request persistence with secret redaction:
  • API keys are not stored in plaintext.
  • Stored payloads include only boolean markers (e.g., api_key_present).
• Artifacts lifecycle management:
  • The worker creates job - local artifacts under LLAMATOR_MCP_ARTIFACTS_ROOT/<job_id>/....
  • Artifacts are uploaded to MinIO as an archive named artifacts.zip.
  • The HTTP API can list available objects under a job prefix and resolve presigned download links.
• Optional API - key protection: For both HTTP and MCP interfaces via X - API - Key.
• OpenAPI schema: With API - key authorization support.
• Prometheus metrics: Exposed at /metrics.

📦 Installation

Requirements

• Docker
• Docker Compose

Start the full stack

docker compose up --build

Default service endpoints

• HTTP API: http://localhost:8000
• MinIO S3 endpoint: http://localhost:9000
• MinIO console: http://localhost:9001

Healthcheck

curl -sS http://localhost:8000/v1/health

📚 Documentation

Configuration

All configuration is provided via environment variables prefixed with LLAMATOR_MCP_. A complete reference is available in DOCUMENTATION.md.

Typical local setup

cp .env.example .env

Key configuration categories

• Redis: Connection DSN for job queue and state storage.
• MinIO: S3 - compatible storage for artifacts.
• Attack/Judge models: OpenAI - compatible endpoints for LLAMATOR execution.
• API security: Optional X - API - Key protection.
• Job execution: Timeouts, TTLs, and retry behavior.

💻 Usage Examples

HTTP API usage

Create a run

curl -sS -X POST "http://localhost:8000/v1/tests/runs" \
  -H "Content-Type: application/json" \
  -H "X-API-Key: <optional>" \
  -d '{
    "tested_model": {
      "kind": "openai",
      "base_url": "http://host.docker.internal:1234/v1",
      "model": "llm",
      "api_key": "lm-studio"
    },
    "run_config": { "enable_reports": false },
    "plan": { "preset_name": "owasp:llm10", "num_threads": 1 }
  }'

The response contains:

• job_id (uuid4 hex, 32 characters)
• status (queued | running | succeeded | failed)
• created_at (UTC timestamp)

Retrieve job state

curl -sS "http://localhost:8000/v1/tests/runs/<job_id>" \
  -H "X-API-Key: <optional>"

Response includes:

• status: current job state
• result: aggregated metrics (when succeeded)
• error: error details (when failed)
• error_notice: compact user - facing error message (when failed)

Artifacts

List objects available for a job:

curl -sS "http://localhost:8000/v1/tests/runs/<job_id>/artifacts" \
  -H "X-API-Key: <optional>"

Resolve a presigned download URL for a specific object:

curl -sS "http://localhost:8000/v1/tests/runs/<job_id>/artifacts/<path>" \
  -H "X-API-Key: <optional>"

The download endpoint returns a JSON payload containing download_url and does not emit redirects.

MCP interface

The MCP server is mounted into the FastAPI application (default mount path: /mcp) and uses Streamable HTTP transport.

Exposed tools

• create_llamator_run: Submits a job, waits for completion, returns aggregated metrics and (if available) a presigned URL for artifacts.zip.
• get_llamator_run: Returns aggregated metrics for a finished job and the optional artifacts archive URL.

Both tools return a consistent response schema:

{
  "job_id": "string",
  "aggregated": {
    "attack_name": {
      "metric": 0
    }
  },
  "artifacts_download_url": "string or null",
  "error_notice": "string or null"
}

Protocol notes, headers, and examples are documented in DOCUMENTATION.md.

🔧 Technical Details

• Security model:
  • If LLAMATOR_MCP_API_KEY is empty, authentication is disabled.
  • If configured, protected HTTP routes and the MCP app require X - API - Key: <value>.
• Local development:
  • Install dependencies:

poetry install

- Run the API server:

uvicorn llamator_mcp_server.main:app --host 0.0.0.0 --port 8000

- Run the worker:

arq llamator_mcp_server.worker_settings.WorkerSettings

• Tutorial: A Jupyter notebook with step - by - step examples is available at notebooks/llamator_mcp_server_tutorial.ipynb. It demonstrates:
  • HTTP API usage with curl
  • MCP JSON - RPC protocol interaction
  • Polling for job completion
  • Artifacts retrieval
• Tests: Integration tests are located in llamator - mcp - server/tests and rely on tests/.env.test.

pytest -q

📄 License

This project is licensed under the terms of the Creative Commons Attribution - ShareAlike 4.0 International license. See the LICENSE file for details.